-- Logs begin at Thu 2022-05-26 12:56:00 UTC, end at Thu 2022-05-26 14:11:10 UTC. -- May 26 13:29:10.642090 dv-cinder-819668-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 13:29:10.651132 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 13:29:10 2022] *** May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: nodename: dv-cinder-819668-4-iscsi May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: machine: x86_64 May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: clock source: unix May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: pcre jit disabled May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: detected number of CPU cores: 4 May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: current working directory: / May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: detected binary path: /usr/bin/uwsgi-core May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: your processes number limit is 47680 May 26 13:29:10.652882 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: your memory page size is 4096 bytes May 26 13:29:10.653500 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: detected max file descriptor number: 2048 May 26 13:29:10.653500 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: lock engine: pthread robust mutexes May 26 13:29:10.653500 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: thunder lock: enabled May 26 13:29:10.653500 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 13:29:10.653500 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 13:29:10.704619 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: Python main interpreter initialized at 0x555fc26149c0 May 26 13:29:10.704619 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: python threads support enabled May 26 13:29:10.704619 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: your server socket listen backlog is limited to 100 connections May 26 13:29:10.704619 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: your mercy for graceful operations on workers is 90 seconds May 26 13:29:10.705046 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: mapped 403077 bytes (393 KB) for 2 cores May 26 13:29:10.705046 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: *** Operational MODE: preforking *** May 26 13:29:10.705367 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: *** uWSGI is running in multiple interpreter mode *** May 26 13:29:10.705367 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: spawned uWSGI master process (pid: 80383) May 26 13:29:10.705573 dv-cinder-819668-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 13:29:10.706183 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: spawned uWSGI worker 1 (pid: 80394, cores: 1) May 26 13:29:10.706539 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: spawned uWSGI worker 2 (pid: 80395, cores: 1) May 26 13:29:10.706687 dv-cinder-819668-4-iscsi devstack@keystone.service[80383]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 13:29:16.470785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:29:16.470785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes May 26 13:29:16.471593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:29:16.471593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes May 26 13:29:16.522159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 13:29:16.522343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 13:29:16.522526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] command line args: [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 13:29:16.522798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 13:29:16.522978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ================================================================================ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 13:29:16.523243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.523494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_dir = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.523686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.524450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_source = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.524640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] debug = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.524853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_config_append = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_dir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.526782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.527036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.527273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.527639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.527891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.528076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.528265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.528538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.528785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.529020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.529255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.529488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.529722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.529970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.530216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.530494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.530995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.531181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] publish_errors = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.531416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.531647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.531829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.532081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.532333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.532575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] standard_threads = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.532807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.533052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.533230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.533408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_journal = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.533586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_json = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.533762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_stderr = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.533938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_syslog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.534166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.534425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.534672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.534925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.535137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:29:16.535137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 26 13:29:16.535657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.535657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.535800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 13:29:16.535800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 26 13:29:16.535939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.535939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.536073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.536335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.536583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.536832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.537066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.537313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.537549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.537794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.538040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.538279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.538533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.538787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.539031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.539265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.539608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.539974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.540233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.540519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.540755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.540988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.541220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.541456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.541709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.541962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.542191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.542422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.542666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.542912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.543150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.543395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.543626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.543862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.544098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.544364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.544578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.544809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.545038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.545273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.545518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.545747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.545982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.546211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.546445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.546679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.546924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.547153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.547387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.547622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.547856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.548088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.548342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.548580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.548820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.549057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.549294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.549525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.549784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.550018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.550247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.550482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.550718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.550988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.551238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.551489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.551736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.551977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.552217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.552477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.552726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.552966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.553204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.553445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.553682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.553919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.554154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.554388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.554625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.554885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.555156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.555529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.555966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.556216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.556481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.556723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.556962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.557197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.557430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.557657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.557953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.558123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.558365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.558601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.558849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.559097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.559355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.559618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.559862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.560096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.external = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.560347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.560644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.560876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.561110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.password = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.561350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.token = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.561582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.561817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.562069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.562301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.562549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.562794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.563023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.563258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.563506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.563756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.564001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.564237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.564509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.564746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.565013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.565268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.565505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.565758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.566046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.566294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.566558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.566802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.567059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.567298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.567544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.567791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.568039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.568267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.568540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.568779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.569031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.569278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.569539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.569775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.570022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.570259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.570507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.571278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.571515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.571807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.571993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.572238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.572554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.572745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.573010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.573310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.573674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.573910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.574159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.574454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.574738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.575008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.575335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.575614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.575858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.576100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.576353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.576617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.576836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.577074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.577319 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.577561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.577810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.578044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.578293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.578541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.578777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.579025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.579274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.579507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.579747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.579979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.580216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.580477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.580725 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.580962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.581200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.581448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.581696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.581929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.582159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.582414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.582651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 13:29:16.582773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.582910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 13:29:16.583031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.583147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] command line args: [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 13:29:16.583263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.583379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 13:29:16.583517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.583632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ================================================================================ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 13:29:16.583751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.583869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.583984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.584101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_dir = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.584230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.584386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.584501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.584883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_source = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.584883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] debug = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.585113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.585240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.585389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.585509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.585625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.585741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.585942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.586052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.586169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.586286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.587003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.587124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.587252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.587366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_config_append = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.587366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.587578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.587578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.587791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_dir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.587916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.588030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.588145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.588261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.588538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.588659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.588776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.588904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.589018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.589135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.589135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.589345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.589458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.589588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.589702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.589821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.589946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.590059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.590188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.590301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.590418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.590534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.590649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.590770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.590898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.591010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.591134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.591248 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.591373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.591498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.591629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.591742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.591742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.591961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.592078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.592205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] publish_errors = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.592397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.592520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.592636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.592753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.592871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.592987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.593116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.593232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.593349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.593466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.593585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.593701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] standard_threads = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.593829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.593944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.593944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.594183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.594304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.594304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_journal = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.594533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.594645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_json = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.594764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.594883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_stderr = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.594883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_syslog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.595100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.595228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 13:29:16.595349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.595476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.595587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.595696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.595817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.595946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596060 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.596978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.597953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.598940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.599061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.599190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.599310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.599700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.599814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.599948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.600806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.601744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.602986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.603989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.604906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.605885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.606983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.607960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.608988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.609976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.610945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.611942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.612938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.613935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.614910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.615644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.616680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.617928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.external = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.618649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.618649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.618649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.618979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.password = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.619095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.token = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.619327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.619562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.619806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.620033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.620279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.620548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.620778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.621011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.621256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.621512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.621750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.621983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.622224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.622458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.622718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.622969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.623202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.623452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.623936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.624402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.624880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.625324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.625671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.625913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.626155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.626405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.626680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.626949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.627252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.627526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.627802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.628038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.628294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.628554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.628802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.629034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.629280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.629526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.629814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.630106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.630351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.630581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.630818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.631065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.631319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.631548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.631777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.632004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.632245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.632518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.632769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.633990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.633990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.634448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.634448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.634699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.634699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.634956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.635156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.635390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.635630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.637281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.637564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.637809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.638068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.638322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.638564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.639574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.639835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.640074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.640397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.640571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.640817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.641057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.641304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.641543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.641779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.642029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.644568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.644838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.645498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.645498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.645766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.645766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.651372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.651979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.653456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.654597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.655589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.656223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.656223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.656223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.657321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.659438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.659438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.659438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.659438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.659769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.660210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.665599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.666089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.666089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.666089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.666299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.666536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.666785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.667033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.667281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.667529 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.667762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.668016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.668264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.668527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.668759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.668989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.669218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.669466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.669736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.669964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.670198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.670430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.670657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.670891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.671122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.671367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.671597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.671849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.672091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.672351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.672597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.672834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.673079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.673313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.673546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.673776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.674017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.674259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.674485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.674723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.675031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.675297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.675540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.675784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.676018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.676256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.676514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.676751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.676984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.677216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.677458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.677691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.677931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.678161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.678404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.678635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.678877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.679106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.679341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.679572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 13:29:16.679740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 13:29:16.687004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.687232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.707998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.708229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.708463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.718271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.737154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.737351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.743478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.776023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.796661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.796945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.815024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.815252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.868434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.868684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.868861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.868992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.869707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.894992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.895124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.895211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.895286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.895361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.895445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.895522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.907261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.907422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.920689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.920827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.920913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.920990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.921950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.922083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 13:29:16.942271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.942458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:29:16.942801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.942947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.943031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:29:16.943109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:29:16.943259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.943438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.943532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.943643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 13:29:16.943755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.943915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.943999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.944174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.944458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.944882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.956149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.956505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.956774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.956886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.957133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.957233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.957360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.957699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.966124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:16.966542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:16.969434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:16.969709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.969885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:16.969989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.970172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.970318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.970453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.970542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.970730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.970828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.971918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.972000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.972180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.972271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.972481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.972750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:16.999578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.000148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.001195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.001608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.001965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.002249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.007103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.007506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.007656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.007788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.007942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.008960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.016024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.016326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.016416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.016609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.021271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.021716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.022002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.022286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.022564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.022839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.023113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.023381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.024405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.024914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.025394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.026045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.026458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.026902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.027463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.027735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.042791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.043082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.047892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.051610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.051909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.053187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.053477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.058092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.058944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.061764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.062597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.062897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.065893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.066196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.066486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.066777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.067058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.067338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.067986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.072517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.076212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.076535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.076958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.077239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.077518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.077794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.088748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.089057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.089326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.089585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.097320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.097616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.097761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.097913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.098120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.099407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.099683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.102933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.103210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.103495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.103776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.104050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.106683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.108423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.108694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.111757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.112032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.112327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.112616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.112893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.118271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.118560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.118848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.119133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.119414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.119689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.119969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.127082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.127373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.127653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.127931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.128209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.128516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.128802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.138845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.142185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.145779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.147516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.147926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.149269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.149614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.149907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.151428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.154767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.156379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.157415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.158020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.158310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.158589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.159041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.162610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.162907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.163192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.163474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.166570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.168186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.171448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.171732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.172017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.172286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.173460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.173755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.174148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.182037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.182326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.182619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.182724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.182987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.183290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.183646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.183920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.184194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.184529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.184803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.185084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.185353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.185621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.185882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.186150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.186419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.191284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 13:29:17.191566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.191845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.192254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.192564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.192841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.193116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.193376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.193659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.193931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.194198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.194582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.194846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.195138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 13:29:17.227525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x555fc26149c0 pid: 80394 (default app) May 26 13:29:17.231547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cb9c2-3541-4f84-aa77-38e58ad30e3c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:17.231838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cb9c2-3541-4f84-aa77-38e58ad30e3c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:17.232127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cb9c2-3541-4f84-aa77-38e58ad30e3c None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:17.236722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu May 26 13:29:17 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:17.238440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x555fc26149c0 pid: 80395 (default app) May 26 13:29:25.336894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:25.337498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:25.337498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:25.338815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:29:25.346619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:25.347114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:25.347295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:25.415650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: INFO dbcounter [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] Registered counter for database keystone May 26 13:29:25.437484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG oslo_db.sqlalchemy.engines [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 13:29:25.443757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writer thread running {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 13:29:25.621746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80395) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 13:29:25.622073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80395) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 13:29:25.677200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:25.678790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:25.806482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1/3] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 26 13:29:25.977535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:26.045841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: INFO dbcounter [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] Registered counter for database keystone May 26 13:29:26.065044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG oslo_db.sqlalchemy.engines [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80394) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 13:29:26.072021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writer thread running {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 13:29:26.193718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a96284-3309-4cbe-ab4f-5f59bcee7763 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:26.198697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:26.199368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:26.199656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:26.199939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:26.208204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:26.262392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 13:29:26.262733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41a7de-f800-4ce7-909a-a91339e55f9f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:26.272997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 3/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu May 26 13:29:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:29:26.279081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:26.293542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe1c7a-a4c4-4d92-8e1a-a889a4c74dc8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:26.297887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:26.299017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:26.299297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:26.299573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] PATH_INFO: `/v3/projects/56278f94460f422387fbeb96048d2270` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:26.301214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] RBAC: Authorizing `identity:get_project(project_id=56278f94460f422387fbeb96048d2270)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:26.354790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING keystone.common.rbac_enforcer.enforcer [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 13:29:26.355407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bf0c8-0400-4ad8-9ada-0daa52ba25b5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:26.359789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 2/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu May 26 13:29:26 2022] GET /identity/v3/projects/56278f94460f422387fbeb96048d2270 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:28.552250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ae142-b495-4503-8eaf-0a1600fd4aa7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:28.552834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ae142-b495-4503-8eaf-0a1600fd4aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:28.552834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ae142-b495-4503-8eaf-0a1600fd4aa7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:28.557595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 4/6] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:28.563797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:28.564265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:28.564386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:28.585125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:28.586709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:28.594662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 13:29:28.617688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ae142-b495-4503-8eaf-0a1600fd4aa7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:28.629653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ae142-b495-4503-8eaf-0a1600fd4aa7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:28.634759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2000e0c-7912-4b19-9ff1-c9d41ff78237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:28.635405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000e0c-7912-4b19-9ff1-c9d41ff78237 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:28.635693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000e0c-7912-4b19-9ff1-c9d41ff78237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:28.635974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2000e0c-7912-4b19-9ff1-c9d41ff78237 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:28.644034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000e0c-7912-4b19-9ff1-c9d41ff78237 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:28.645858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2000e0c-7912-4b19-9ff1-c9d41ff78237 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:28.654757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu May 26 13:29:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:29:28.660541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:28.669618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba340f4-f079-4802-8753-86cb6eec540a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:28.674051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa605c-9046-425d-a9d4-0222828a2328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:28.674922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa605c-9046-425d-a9d4-0222828a2328 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:28.675210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa605c-9046-425d-a9d4-0222828a2328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:28.675492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa605c-9046-425d-a9d4-0222828a2328 admin admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:28.676921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa605c-9046-425d-a9d4-0222828a2328 admin admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:28.678767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa605c-9046-425d-a9d4-0222828a2328 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:28.681069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 4/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu May 26 13:29:28 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:30.975270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de83b41-7571-476c-9ab6-794b0b17f2d9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:30.975802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de83b41-7571-476c-9ab6-794b0b17f2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:30.975802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de83b41-7571-476c-9ab6-794b0b17f2d9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:30.977523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:30.982945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:30.983409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:30.983565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.001582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:31.002765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:31.069832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e40322-ca98-430e-8703-98c00e0e90d4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.071461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e40322-ca98-430e-8703-98c00e0e90d4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.071900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e40322-ca98-430e-8703-98c00e0e90d4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.073627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 7/11] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:31.087930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.088220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.088709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.091698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 5/12] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:31.115141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.131794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80394) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 13:29:31.132220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80394) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 13:29:31.140721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:31.179416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:31.181658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:31.235498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54542d-8a13-41c1-a946-65839e828e7f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:31.271938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:31.272942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.273129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.273377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.294638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 8/13] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:29:31.317724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.399315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:31.411166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:31.412287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:31.413167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 6/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:29:31.419860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.509771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:31.529945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bf560-3556-482a-812e-4d3a6b209ead None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:31.542292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e755b-5d22-4540-8b83-791c17b0ec74 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:31.544860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:31.569268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:31.570014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.570309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.570595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.571355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:31.571674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:31.573142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:31.586390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 7/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:31.587717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:31.588773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.589071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.589354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.591989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.661073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:31.662688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:31.698521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34604ec-61df-4704-b3df-015ce0bd2c11 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:31.735880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:31.736805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.737090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.737377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.738642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:31.740142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:31.741832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 8/16] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:31.747477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.763820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:31.776830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 9/17] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:29:31.782566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.786252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:31.859504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb7c57-b066-4944-b001-bfdfb21f5c5d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:31.887817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12cc8-c646-4879-a552-da2a488c2dfb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:31.897877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:31.899271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.899561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.899839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.901135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:31.902621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:31.904527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:31.912028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:31.931621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:31.932471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:31.932759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:31.933036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:31.952967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:31.954750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:29:31.976830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:31.978480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:31.979570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:29:31.979570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:29:31.979570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:29:31.979570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:31.979570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:29:31.983972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:32.001924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:32.022758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5da4-d84d-41e8-aa0f-ce5fd6c454ab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:32.046972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:32.063972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:32.064716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:32.064975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:32.065257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:32.066111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.domain.id=default, user.id=8243f73b32484519bc2bed2bc9edc57e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:32.066422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:32.067917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efd516-321f-4460-ba23-9c82140b1f5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:32.077332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1322 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/role_assignments?user.id=8243f73b32484519bc2bed2bc9edc57e&scope.domain.id=default&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:32.096700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a0ae2-ee35-482c-a81e-81cfbc2d9cfb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:32.140337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:32.141036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:32.141325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:32.141597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:32.142346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:32.142648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:32.144096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f420c-efff-429c-bb5a-65442bfb96e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:32.165141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:34.501947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40d2b-1cab-471b-8ef9-9e11b7069247 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:34.501947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40d2b-1cab-471b-8ef9-9e11b7069247 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:34.501947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40d2b-1cab-471b-8ef9-9e11b7069247 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:34.501947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:34.507254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:34.508233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:34.508474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:34.533726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:34.534923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:34.643806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:34.673616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40d2b-1cab-471b-8ef9-9e11b7069247 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:34.839998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40d2b-1cab-471b-8ef9-9e11b7069247 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:34.889993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:34.891166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:34.891680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:34.892172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:34.909395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:34.918425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:34.919255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:34.929944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 12/24] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:34.932246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:35.018246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:35.108416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48b34a-c251-43ca-a9bd-188163bf4821 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:35.150819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:35.151511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.151802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.152085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.152872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:35.153182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:35.154585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c61f9-454a-47e1-ab4a-5edbda500b68 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:35.176604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 13/25] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:35.182949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:35.193723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:35.193905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2072a-abc0-473e-96f3-e4d7763ae1ef None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.194120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2072a-abc0-473e-96f3-e4d7763ae1ef None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.194377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2072a-abc0-473e-96f3-e4d7763ae1ef None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.196014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 14/26] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:35.201767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170162b2-7cf2-47fc-94d2-2889a52897d7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.201904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170162b2-7cf2-47fc-94d2-2889a52897d7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.202085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170162b2-7cf2-47fc-94d2-2889a52897d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.230282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-170162b2-7cf2-47fc-94d2-2889a52897d7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:35.231552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-170162b2-7cf2-47fc-94d2-2889a52897d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:35.417689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3f1b2-98dd-46c0-a9ce-0736d339f9fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:35.437948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 15/27] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:29:35.443291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b39b7-7da8-4c5e-9357-89e2011068d1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.443796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b39b7-7da8-4c5e-9357-89e2011068d1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.444223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b39b7-7da8-4c5e-9357-89e2011068d1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.445796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 16/28] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:29:35.457855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.458433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.458938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.465797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:35.466680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.466965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.467244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.473103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:35.476961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:35.479106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 13/29] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:35 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:35.482218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:35.483543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:35.485020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:35.542936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:35.568067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:35.595456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 17/30] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:29:35.596684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:35.602325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f63be-384a-4077-a688-e1d269a5aa21 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:35.643044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:35.643044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.643044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.643044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.656567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:35.658951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b57b5e12-8f01-40c6-b18a-233801da5f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:29:35.686485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:35.687330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:35.688140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:29:35.688140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:29:35.688140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:29:35.688140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:35.689886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 14/31] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:35.695922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55afd-fa07-40c0-b034-10497a5e91f9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.695922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55afd-fa07-40c0-b034-10497a5e91f9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.695922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55afd-fa07-40c0-b034-10497a5e91f9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.695922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 15/32] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:35.695922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a55afd-fa07-40c0-b034-10497a5e91f9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:35.721410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0d152-8424-4507-bf3f-abfb6927d078 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:35.764920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:35.766199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.766769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.767179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.769352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:35.771403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:35.774405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 18/33] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:35.775627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:35.814599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:35.819586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a55afd-fa07-40c0-b034-10497a5e91f9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:35.846318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:35.860412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:35.861281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.861663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.862041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.862890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:35.864546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:35.888373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e33e-0774-4673-80b1-49161f2f7d54 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:35.937191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:35.938157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.938624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.939088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.939982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:35.940517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:35.942131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150e361-1d9e-49d2-80ea-ae1757c1c4b1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:35.955534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 19/34] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:35.957169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643147c-641b-4629-a317-8100a971ed12 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:35.957605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643147c-641b-4629-a317-8100a971ed12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:35.958052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643147c-641b-4629-a317-8100a971ed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:35.976362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e643147c-641b-4629-a317-8100a971ed12 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:35.978361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e643147c-641b-4629-a317-8100a971ed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:36.010819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 16/35] 10.222.0.16 () {60 vars in 1131 bytes} [Thu May 26 13:29:35 2022] POST /identity/v3/roles => generated 225 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:36.012197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:36.055590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 20/36] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:36.056777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643147c-641b-4629-a317-8100a971ed12 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:36.088647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:36.143248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98a096-a818-4722-849e-b4af2a5204ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:36.187060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:36.188129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.188451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.188809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] PATH_INFO: `/v3/domains/default/users/8243f73b32484519bc2bed2bc9edc57e/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.191437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8243f73b32484519bc2bed2bc9edc57e, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:36.193239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:36.204033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643147c-641b-4629-a317-8100a971ed12 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:36.215032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 17/37] 10.222.0.16 () {58 vars in 1361 bytes} [Thu May 26 13:29:36 2022] PUT /identity/v3/domains/default/users/8243f73b32484519bc2bed2bc9edc57e/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:29:36.216263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:36.249341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef82ac4-28ee-4811-ba65-f5b08fce16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:36.250092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef82ac4-28ee-4811-ba65-f5b08fce16b8 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.250384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef82ac4-28ee-4811-ba65-f5b08fce16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.250666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef82ac4-28ee-4811-ba65-f5b08fce16b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.251506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef82ac4-28ee-4811-ba65-f5b08fce16b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:36.253126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef82ac4-28ee-4811-ba65-f5b08fce16b8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:36.293371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:36.371327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0e366-0ee8-4d15-99e1-05d91ab6efe2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:36.410388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39eee3-aec1-4628-a6c7-001f8746494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:36.411422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39eee3-aec1-4628-a6c7-001f8746494c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.411931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39eee3-aec1-4628-a6c7-001f8746494c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.412538 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39eee3-aec1-4628-a6c7-001f8746494c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.414061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39eee3-aec1-4628-a6c7-001f8746494c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:36.416276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39eee3-aec1-4628-a6c7-001f8746494c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:36.432350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 18/38] 10.222.0.16 () {60 vars in 1131 bytes} [Thu May 26 13:29:36 2022] POST /identity/v3/roles => generated 219 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:36.485765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 21/39] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:29:36 2022] POST /identity/v3/projects => generated 301 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:36.781301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bafd13-f18c-4157-99ed-c7136fb85aa9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.782276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bafd13-f18c-4157-99ed-c7136fb85aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.782783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bafd13-f18c-4157-99ed-c7136fb85aa9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.785369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 19/40] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:36.792035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c56f0d-ceb2-43d8-86a2-6f8e939421eb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.792992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c56f0d-ceb2-43d8-86a2-6f8e939421eb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.793513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c56f0d-ceb2-43d8-86a2-6f8e939421eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.818524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9c56f0d-ceb2-43d8-86a2-6f8e939421eb None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:36.820776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9c56f0d-ceb2-43d8-86a2-6f8e939421eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:36.900199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 22/41] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:36.905445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ed34c-975b-42e3-a190-079384860a41 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.905739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ed34c-975b-42e3-a190-079384860a41 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.906019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ed34c-975b-42e3-a190-079384860a41 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.907392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 20/42] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:36.912932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:36.913183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:36.913787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:36.924292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ed34c-975b-42e3-a190-079384860a41 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:36.934095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:36.935296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:36.937492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:37.016740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 23/43] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:37.037363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ed34c-975b-42e3-a190-079384860a41 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:37.040694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:37.040818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:37.098560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:37.099368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:37.099651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:37.100013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:37.101551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:37.103112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:37.105014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 21/44] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:37.110418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:37.176937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99fe9b-76d8-4542-b58d-9345ce20f7d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:37.223749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ad980-74bf-4ad7-b9f2-2a65958c4630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:37.224848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ad980-74bf-4ad7-b9f2-2a65958c4630 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:37.225346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ad980-74bf-4ad7-b9f2-2a65958c4630 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:37.225794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ad980-74bf-4ad7-b9f2-2a65958c4630 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:37.227377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ad980-74bf-4ad7-b9f2-2a65958c4630 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:37.228974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ad980-74bf-4ad7-b9f2-2a65958c4630 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:37.235573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f55bc2-43ad-42c2-a005-0a14fc4a7f07 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:37.240052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:37.252641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 24/45] 10.222.0.16 () {60 vars in 1131 bytes} [Thu May 26 13:29:37 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:37.285839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eb644-9d7c-473b-abcf-d5224893a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:37.286615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eb644-9d7c-473b-abcf-d5224893a6a2 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:37.286971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eb644-9d7c-473b-abcf-d5224893a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:37.287316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eb644-9d7c-473b-abcf-d5224893a6a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:37.288211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9eb644-9d7c-473b-abcf-d5224893a6a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:37.290415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9eb644-9d7c-473b-abcf-d5224893a6a2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:37.322440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 22/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:29:37 2022] POST /identity/v3/projects => generated 312 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:29:38.675601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce967ffd-3fba-429e-b456-d06f7dd7415f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:38.676120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce967ffd-3fba-429e-b456-d06f7dd7415f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:38.676239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce967ffd-3fba-429e-b456-d06f7dd7415f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:38.677594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 25/47] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:38.683509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:38.683911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:38.684079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:38.701284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:38.702786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:38.723652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:38.775206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 23/48] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:38.798078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce967ffd-3fba-429e-b456-d06f7dd7415f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:38.903552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce967ffd-3fba-429e-b456-d06f7dd7415f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:38.940849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:38.941647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:38.941956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:38.942218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:38.954486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:38.964137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:38.965071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:38.965826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 26/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:38.971115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:39.072861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cab4a9-ca47-4c97-87ae-3274f867c848 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:39.109802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:39.110482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.110765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.111044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.111793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:39.112103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:39.113531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:39.122801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 24/50] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:39.128488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:39.187539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:39.232259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa063bb0-0ce0-4723-ac2e-0b3c2d8e77b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:39.267018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:39.267899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.268188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.268508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.269945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:39.271397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:39.273198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 27/51] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:39 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:39.278618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:39.296884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:39.383836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f68dd3-a1d8-461b-957e-92dfc28cb2ac None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:39.428973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19f16-ded5-4018-9a5e-46b54f0a3c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:39.429759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19f16-ded5-4018-9a5e-46b54f0a3c08 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.430047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19f16-ded5-4018-9a5e-46b54f0a3c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.430322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19f16-ded5-4018-9a5e-46b54f0a3c08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.431510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19f16-ded5-4018-9a5e-46b54f0a3c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:39.433084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19f16-ded5-4018-9a5e-46b54f0a3c08 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:39.434731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 25/52] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:39.440629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:39.560150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1628b-e9f0-41df-9da1-ac0221048ac7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:39.587721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:39.590592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539b04-43d7-43fe-b257-e81707bcc4b6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.590970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539b04-43d7-43fe-b257-e81707bcc4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.591320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539b04-43d7-43fe-b257-e81707bcc4b6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.592830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 26/53] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:39.598340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.598630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.598914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.600235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:39.601425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.601723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.602016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.602878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.domain.id=default, user.id=8243f73b32484519bc2bed2bc9edc57e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:39.603200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:39.604707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:39.614322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 28/54] 10.222.0.16 () {56 vars in 1322 bytes} [Thu May 26 13:29:39 2022] GET /identity/v3/role_assignments?user.id=8243f73b32484519bc2bed2bc9edc57e&scope.domain.id=default&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 530 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:39.616262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:39.617529 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:39.696354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 27/55] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:29:39.720270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:39.845146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f96c2-b379-4b2f-8d12-987a162916af None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:39.880626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6448690-99f1-4d76-9f20-49c45ec0d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:39.881431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6448690-99f1-4d76-9f20-49c45ec0d858 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:39.881724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6448690-99f1-4d76-9f20-49c45ec0d858 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:39.882075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6448690-99f1-4d76-9f20-49c45ec0d858 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:39.884078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6448690-99f1-4d76-9f20-49c45ec0d858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:39.885591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6448690-99f1-4d76-9f20-49c45ec0d858 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:39.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:39.893288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:39.909147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:39.995172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c46919-ff1e-4293-ad76-7602ed7dad53 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:40.029038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be382473-348d-42b2-9734-b3adff4009c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:40.029709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be382473-348d-42b2-9734-b3adff4009c8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:40.030000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be382473-348d-42b2-9734-b3adff4009c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:40.030286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be382473-348d-42b2-9734-b3adff4009c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:40.031014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be382473-348d-42b2-9734-b3adff4009c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:40.032524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be382473-348d-42b2-9734-b3adff4009c8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:40.057580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu May 26 13:29:39 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:42.292608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fae8d-676c-4640-b893-d4a5a9df8bea None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:42.293119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fae8d-676c-4640-b893-d4a5a9df8bea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:42.293205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fae8d-676c-4640-b893-d4a5a9df8bea None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:42.294583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:42.299838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:42.300238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:42.300431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:42.317565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:42.318761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:42.380863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:42.389546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:42.413957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fae8d-676c-4640-b893-d4a5a9df8bea None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:42.471802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:42.517509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fae8d-676c-4640-b893-d4a5a9df8bea None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:42.550022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3882e4-2020-44e7-8159-26d3961d6c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:42.550749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3882e4-2020-44e7-8159-26d3961d6c99 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:42.551039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3882e4-2020-44e7-8159-26d3961d6c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:42.551324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3882e4-2020-44e7-8159-26d3961d6c99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:42.552553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3882e4-2020-44e7-8159-26d3961d6c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:42.554625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3882e4-2020-44e7-8159-26d3961d6c99 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:42.556273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:42.561857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:42.657465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a281918-50f9-41cb-8231-9502d1df5126 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:42.693630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac550b5-7e4c-470d-b4fd-656fd90b4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:42.694335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac550b5-7e4c-470d-b4fd-656fd90b4d57 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:42.694634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac550b5-7e4c-470d-b4fd-656fd90b4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:42.694921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac550b5-7e4c-470d-b4fd-656fd90b4d57 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:42.695648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac550b5-7e4c-470d-b4fd-656fd90b4d57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:42.697154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac550b5-7e4c-470d-b4fd-656fd90b4d57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:42.722374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:29:42 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:46.217361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ce5b5-6604-48fe-b121-ab09c1812a18 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.218758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ce5b5-6604-48fe-b121-ab09c1812a18 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.219497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ce5b5-6604-48fe-b121-ab09c1812a18 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.221294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:46.242438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f46e2-7681-4681-81e1-32acada3f6bd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.242950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f46e2-7681-4681-81e1-32acada3f6bd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.243206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f46e2-7681-4681-81e1-32acada3f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.268665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-667f46e2-7681-4681-81e1-32acada3f6bd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:46.269875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-667f46e2-7681-4681-81e1-32acada3f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:46.321966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd439c-4069-473b-a830-3d4c6f12ce33 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.322283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd439c-4069-473b-a830-3d4c6f12ce33 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.322528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd439c-4069-473b-a830-3d4c6f12ce33 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.323926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 33/63] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:46.330628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70bfeb-c8b3-400c-9544-91d0f942bd9b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.331201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70bfeb-c8b3-400c-9544-91d0f942bd9b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.331689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70bfeb-c8b3-400c-9544-91d0f942bd9b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.333300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 34/64] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:29:46.337582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.338145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.338634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.340282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:46.357418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:46.359136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:46.366175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 31/65] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:46.367512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.367812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.368104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.385305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:46.386512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:46.435568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 35/66] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:46.444413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:46.457432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 32/67] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:46.473266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:46.533985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:46.584045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd740a0-780a-4936-9796-0c9b000082c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:46.585392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c69db-5a7b-4409-b409-3f424a14f112 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:46.599433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:46.623442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:46.623700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:46.624714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.624848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.625238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.625407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.625700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.625988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.637996 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:46.638383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:46.646524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:46.648027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:46.648851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:46.649570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:46.650388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:46.651160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 36/68] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:46.651799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:46.654311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 33/69] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:29:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:46.660506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:46.774902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c32201-3189-4536-ad05-e340ecdd354d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:46.802699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c499888-10e4-4e88-b17c-d00277707722 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:46.821337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:46.822986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:46.823616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.823911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.824687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.825434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:46.825434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:46.827431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:46.839691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 34/70] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:46.841050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:46.846120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdf1bb-8b21-4283-9c40-b55545fe34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:46.847266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdf1bb-8b21-4283-9c40-b55545fe34e5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.847559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdf1bb-8b21-4283-9c40-b55545fe34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.847843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdf1bb-8b21-4283-9c40-b55545fe34e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.862491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdf1bb-8b21-4283-9c40-b55545fe34e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-32fdf1bb-8b21-4283-9c40-b55545fe34e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:46.872916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:46.873924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:46.874701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 37/71] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:29:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:46.875394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec35a63-abc4-4d4f-ae5e-e312da7ac40f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.875688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec35a63-abc4-4d4f-ae5e-e312da7ac40f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.875966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec35a63-abc4-4d4f-ae5e-e312da7ac40f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.879111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 38/72] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:46.879111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec35a63-abc4-4d4f-ae5e-e312da7ac40f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:46.884845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:46.953629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a436c-b729-4d34-ae58-2ab2d751deaf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:46.986127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec35a63-abc4-4d4f-ae5e-e312da7ac40f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:46.987528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:46.988200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:46.988523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:46.988808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:46.989551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:46.989862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:46.991822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e430ba-6285-4983-9b5e-30e85e97ba92 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.000244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 35/73] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.001347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c914024-457c-4995-9e6d-bf252db42661 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.001632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c914024-457c-4995-9e6d-bf252db42661 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.001920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c914024-457c-4995-9e6d-bf252db42661 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.003180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 36/74] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:47.003885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c914024-457c-4995-9e6d-bf252db42661 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.026476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab07f1-142c-427c-9f62-21f06aa4c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.027283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab07f1-142c-427c-9f62-21f06aa4c6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.027581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab07f1-142c-427c-9f62-21f06aa4c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.027868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab07f1-142c-427c-9f62-21f06aa4c6a7 None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.043726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab07f1-142c-427c-9f62-21f06aa4c6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.045241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab07f1-142c-427c-9f62-21f06aa4c6a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.046941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 39/75] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:46 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.048146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.048461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.048749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.064911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:47.066121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:47.094707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.104338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.111038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c914024-457c-4995-9e6d-bf252db42661 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.136275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 40/76] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:47.141181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.144394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.145107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.145472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.145765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.146558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:47.146877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.149014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1492d-f0c6-4e6d-a73e-14a3cbf498b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.162407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 37/77] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.162407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.162407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.162407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.178014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:47.179830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:47.245687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe4ff6-c795-4cf0-9ccb-bf2b2a54e324 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.250560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 38/78] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:47.251607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.289189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.289990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.290281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.290565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.291788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.293312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.294972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 41/79] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.295742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.328807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.345825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.359686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201c8e5-5711-454a-9ed0-b728c0bb5def None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.393643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.394290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.394575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.394853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.405644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3e7c7-ce13-4317-a592-ce75f4411094 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.443615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.444556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.444856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.445141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.455410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:47.463990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:47.464960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:47.465738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 42/80] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:47.466735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.466806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.467021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.557285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.562570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.571527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 39/81] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:29:47.572401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.573055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee81b-1689-4c90-9556-cf308e8bd8fc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.608608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.609493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.609789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.610071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.611288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.612791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.614563 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 43/82] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.615252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.674830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cda783-2a04-4935-bc98-d34b15e5fdca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.709134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.710281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.710584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.710846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.712014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.713514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.715306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 40/83] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.715428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41143548-c879-41ca-8426-bbd5796a22fb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.715964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.753236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.754010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.754296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.754579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.775443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.783133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:47.824124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef01c11-da0a-4520-9ef0-e9cd5cdb0282 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:47.836864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.838382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.859613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:47.860578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:47.861008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:47.861445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:47.862200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:47.862514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:47.863937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:47.881241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 41/84] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:47.882148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:47.948558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 44/85] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:29:47.949430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.001402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c904308-83e9-42af-8b9a-7cce12670336 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.012265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.020451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.043711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.044629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.044954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.045248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.046474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.047938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.049678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 42/86] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:48.050436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.059344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf9fb3-d167-464a-96c7-bdbe8ae5241d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.094025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.094436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.094776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.095096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.151420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca41b6a-5eeb-4329-ad21-85d65d4d3391 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.166623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.172104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.192103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.192818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.193111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.193398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.194179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.196169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.215798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 43/87] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:29:48 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:48.216672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.242441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.269559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.298161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 45/88] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:29:48.299677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.345410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be349adf-807e-4b5d-8a71-5a375ceae6fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.387345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.388662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.388952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.389236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.390525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.392044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.393777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 44/89] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:48.394574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.422836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfdaa2-d62c-4816-a845-04da6b95770b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.463823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.464984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.465276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.465549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.466774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.468280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.470219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 46/90] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:48.470784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.510653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.524927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9228fc-3b04-4692-a80c-31bb84aae7f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.538828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.572641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.573316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.573609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.573893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.574927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=a2bb088e25f642dba7a1e5a300b90359) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:48.575059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.576581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.582521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 45/91] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/role_assignments?user.id=a2bb088e25f642dba7a1e5a300b90359&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=ee34d08904734afdb4939fd27d04b760 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:48.583348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.626627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7402d-5e98-4ca3-a4cf-1878cab5825b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.689176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.690089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.690383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.690673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.692033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.693555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.695588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 47/92] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:48.696231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:48.708571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c9140-4832-4af7-871e-3ef2d339b28b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.746674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.748387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.748387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.748555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.800905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.830894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.832872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.904055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657e3ec-1b46-4cb7-8d50-57891f8feb44 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:48.908954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:48.947218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:48.948182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:48.948780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:48.949262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:48.950801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=8243f73b32484519bc2bed2bc9edc57e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:48.951335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:48.953041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:48.961448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 48/93] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/role_assignments?user.id=8243f73b32484519bc2bed2bc9edc57e&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:48.962630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:49.052349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 46/94] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:29:49.058929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:49.152909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5512c22-ccec-44f7-8f21-be7d4cb83d04 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:49.212792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:49.229131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:49.229832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:49.230125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:49.230430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:49.231307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] RBAC: Adding query filter params (role.id=9508980101c44da48df022abc45e1e60, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=a2bb088e25f642dba7a1e5a300b90359) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:49.231632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:49.233399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:49.246659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 49/95] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:29:48 2022] GET /identity/v3/role_assignments?user.id=a2bb088e25f642dba7a1e5a300b90359&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=9508980101c44da48df022abc45e1e60 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:49.246847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4639d8f-59d0-4c00-8717-146e2ab08fed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:49.286238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44400ad8-04e4-4c8d-87df-13fc6d91cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:49.287110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44400ad8-04e4-4c8d-87df-13fc6d91cf7d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:49.287385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44400ad8-04e4-4c8d-87df-13fc6d91cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:49.287665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44400ad8-04e4-4c8d-87df-13fc6d91cf7d None admin] PATH_INFO: `/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:49.289627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44400ad8-04e4-4c8d-87df-13fc6d91cf7d None admin] RBAC: Authorizing `identity:get_project(project_id=69ebd561e1bf46d5b4fc1e44252f986e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:49.291082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44400ad8-04e4-4c8d-87df-13fc6d91cf7d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:49.292836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:49 2022] GET /identity/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:49.298915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:49.336699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:49.443436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2fdd4-fd05-4d24-b561-3b99f0c52d6b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:49.492044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:49.493118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:49.493579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:49.494030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:49.495041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=69ebd561e1bf46d5b4fc1e44252f986e, user.id=a2bb088e25f642dba7a1e5a300b90359) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:49.495549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:49.497449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93114813-c8c3-4356-8c9a-ff4d2cfe5a0e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:49.509650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:29:49 2022] GET /identity/v3/role_assignments?user.id=a2bb088e25f642dba7a1e5a300b90359&scope.project.id=69ebd561e1bf46d5b4fc1e44252f986e&role.id=ee34d08904734afdb4939fd27d04b760 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:51.442665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e58bf-a6f8-4d99-9f9d-91d75c04cd5b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:51.443235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e58bf-a6f8-4d99-9f9d-91d75c04cd5b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:51.443235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e58bf-a6f8-4d99-9f9d-91d75c04cd5b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:51.445390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:51.451171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:51.451668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:51.451758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:51.477362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:51.478577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:51.558320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 51/99] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:51.597346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e58bf-a6f8-4d99-9f9d-91d75c04cd5b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:51.745386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e58bf-a6f8-4d99-9f9d-91d75c04cd5b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:51.783375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:51.784176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:51.784490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:51.784769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:51.795874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:51.804313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:51.805273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:51.806069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 49/100] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:51.812611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:51.864843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:51.957755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571c909-7404-492f-8dcc-a61637793e43 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:52.004224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:52.004884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.005181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.005473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.006273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:52.006658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:52.008171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02236e-4e5a-498c-a727-c59089db79ff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:52.023169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 52/101] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:52.024458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:52.036714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:52.186710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307a6c8-057c-4011-bee4-18bb02a4cf1e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:52.201996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce170e2-72c4-4696-bdcc-205b03958f9d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.202553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce170e2-72c4-4696-bdcc-205b03958f9d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.203001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce170e2-72c4-4696-bdcc-205b03958f9d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.204585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 53/102] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:52.213895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30ae7-35c9-4a10-8afb-1f0bc32bb4f6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.214542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30ae7-35c9-4a10-8afb-1f0bc32bb4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.215042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e30ae7-35c9-4a10-8afb-1f0bc32bb4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.248333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52a39e-4d45-460f-b50b-2e428d95bd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:52.248333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a39e-4d45-460f-b50b-2e428d95bd22 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.248333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a39e-4d45-460f-b50b-2e428d95bd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.248333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a39e-4d45-460f-b50b-2e428d95bd22 None admin] PATH_INFO: `/v3/users/2d38e8d963004c2280138a3d42b00503` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.253572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-62e30ae7-35c9-4a10-8afb-1f0bc32bb4f6 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:52.255092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-62e30ae7-35c9-4a10-8afb-1f0bc32bb4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:52.278886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52a39e-4d45-460f-b50b-2e428d95bd22 None admin] RBAC: Authorizing `identity:get_user(user_id=2d38e8d963004c2280138a3d42b00503)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:52.280384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52a39e-4d45-460f-b50b-2e428d95bd22 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:52.282417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 50/103] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:52 2022] GET /identity/v3/users/2d38e8d963004c2280138a3d42b00503 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:52.283699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf89a-61e8-497c-8563-e2d8e78a5b8c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.283989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf89a-61e8-497c-8563-e2d8e78a5b8c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.284268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf89a-61e8-497c-8563-e2d8e78a5b8c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.285793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 51/104] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:52.290601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf89a-61e8-497c-8563-e2d8e78a5b8c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:52.388818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 54/105] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:52.392580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.394295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.395419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.438643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:52.443645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:52.445199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:52.449629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:52.486072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf89a-61e8-497c-8563-e2d8e78a5b8c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:52.529711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976778e-d1bf-4c06-8cd5-9965a3adfc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:52.530583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976778e-d1bf-4c06-8cd5-9965a3adfc43 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.530876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976778e-d1bf-4c06-8cd5-9965a3adfc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.531310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976778e-d1bf-4c06-8cd5-9965a3adfc43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.536786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 55/106] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:52.537821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:52.612645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976778e-d1bf-4c06-8cd5-9965a3adfc43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:52.614152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976778e-d1bf-4c06-8cd5-9965a3adfc43 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:52.650366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37907d-7a49-487f-8876-aa43608c81f1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:52.687254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:52.690562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c0c8f-db86-4046-bdb4-1f7850b6c189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:52.691702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0c8f-db86-4046-bdb4-1f7850b6c189 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.692260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0c8f-db86-4046-bdb4-1f7850b6c189 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.692595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0c8f-db86-4046-bdb4-1f7850b6c189 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.703289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:52.707762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c0c8f-db86-4046-bdb4-1f7850b6c189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7c0c0c8f-db86-4046-bdb4-1f7850b6c189 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:52.716735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:52.717826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 56/107] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:52.724477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54d8af-71ef-4438-a61f-b522389640d2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54d8af-71ef-4438-a61f-b522389640d2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54d8af-71ef-4438-a61f-b522389640d2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 57/108] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24cceb-9925-407c-905d-ae7343ba3fab None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24cceb-9925-407c-905d-ae7343ba3fab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24cceb-9925-407c-905d-ae7343ba3fab None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.725871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 58/109] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:52.726345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24cceb-9925-407c-905d-ae7343ba3fab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:52.745957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 52/110] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:29:52.747680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b7cd-5e40-4b71-ad67-fdf969a40967 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.747981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b7cd-5e40-4b71-ad67-fdf969a40967 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.748269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b7cd-5e40-4b71-ad67-fdf969a40967 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.749602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 53/111] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:52.752461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2b7cd-5e40-4b71-ad67-fdf969a40967 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:52.857364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24cceb-9925-407c-905d-ae7343ba3fab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:52.859155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2b7cd-5e40-4b71-ad67-fdf969a40967 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:52.893133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14397bd9-73b6-48c9-b192-e59530318277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:52.893937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14397bd9-73b6-48c9-b192-e59530318277 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.894223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14397bd9-73b6-48c9-b192-e59530318277 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.894500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14397bd9-73b6-48c9-b192-e59530318277 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.905205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:52.905542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14397bd9-73b6-48c9-b192-e59530318277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:52.905906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.906192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.906476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.907236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:52.907600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:52.909514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa4381-5137-43e3-a34f-e24dbf9d25d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-14397bd9-73b6-48c9-b192-e59530318277 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:52.914400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:52.915313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:52.916079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:52.917266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 59/112] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:52.917905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.918071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.918336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.920938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 54/113] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:52.926259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf995597-cbc8-4437-87da-b4a66aa9bbc9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:52.926550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf995597-cbc8-4437-87da-b4a66aa9bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:52.926831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf995597-cbc8-4437-87da-b4a66aa9bbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:52.935856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:52.937090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:52.944470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf995597-cbc8-4437-87da-b4a66aa9bbc9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:52.945782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf995597-cbc8-4437-87da-b4a66aa9bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:52.963159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:52.996100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.008405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 60/114] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:53.009249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.018148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 55/115] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:53.019444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.019740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.020026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.040853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:53.042057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:53.109542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 56/116] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:53.110209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.128095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c9a09-3c39-4669-a3cf-7f8df068fc0a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.163151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.164114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.164438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.164725 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.165938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.167414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.169151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 61/117] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.169921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.193834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.214000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde41e49-0573-43e3-83d8-85c17b44d77c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.233736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.248632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.249302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.249587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.249867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.250616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:53.250921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.252347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.260990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 57/118] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.261820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.277935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de72d38-cade-48ab-b773-a1b05b777866 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.312932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.313809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.314115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.314394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.315643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.317161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.323121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 62/119] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.323121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.369720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e61f2c-234c-4a73-afd9-fecdcdeb850c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.403209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.404000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.404286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.404592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.414556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:53.422127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:53.423181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:53.423962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:53.424638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17530083-5627-4e3b-befe-037e9720ac69 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.424710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 58/120] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:53.424710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.429373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.450745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.461280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.462086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.462372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.462649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.463853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.465345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.467121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 63/121] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.467807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.527908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478b97-33eb-4063-ad4a-8ace123ea61b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.561316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.562110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.562397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.562681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.572040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1b806-fd8b-46d5-9f6f-e84021cee2d9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.573340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:53.581079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:53.581960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:53.582737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:53.583372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.608584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.609255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.609546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.609831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.610666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] RBAC: Adding query filter params (role.id=c71df34c0f1c4a2a8f0bff66914681f7, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=2d38e8d963004c2280138a3d42b00503) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:53.610985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.612481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.618119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 64/123] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/role_assignments?user.id=2d38e8d963004c2280138a3d42b00503&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=c71df34c0f1c4a2a8f0bff66914681f7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.618945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.653310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.698999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.706156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dd332-7585-421c-8531-e47a23f3cedf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.744032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.744990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.745285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.745574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.746993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.748548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.750292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 60/124] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.751109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.762442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe072d9-7885-443b-9cdf-acf2e5002a55 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.804123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.805189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.805568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.805966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.807322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.808937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.810744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 65/125] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.813196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.869051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da4dae-42ae-46f3-8f09-c0066df05a1a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.904704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.911755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.913709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.914297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.914751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.915761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:53.916273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.919182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:53.929516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 61/126] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:53.930727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:53.936427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259bcba-8e19-4494-9a62-ff1046625937 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:53.967416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:53.979599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:53.980802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:53.981278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:53.981808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:53.982795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:53.984490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.017361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 66/127] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:29:53 2022] POST /identity/v3/projects => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:29:54.029503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.049735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e59c55-25ff-4da6-9da2-435f3eab8923 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.091095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.091747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.092032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.092333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.093454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:54.093774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.095302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.106112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:54.111822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.175183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:54.176624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb5716-6419-42e5-9fa8-a82b6767edd3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.227065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.227950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.228241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.228552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.230002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.231818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.233912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 67/129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:54 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:54.234951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.243296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398ccd-c163-4e11-9c1d-1b0e3525440d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.283970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.284922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.285209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.285636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/a2bb088e25f642dba7a1e5a300b90359/roles/9508980101c44da48df022abc45e1e60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.288497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] RBAC: Authorizing `identity:create_grant(project_id=b58b1b03bb4947b4be69f8dc27be3cb4, user_id=a2bb088e25f642dba7a1e5a300b90359, role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.290044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.294208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:54.308560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 63/130] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:29:54 2022] PUT /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/a2bb088e25f642dba7a1e5a300b90359/roles/9508980101c44da48df022abc45e1e60 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:29:54.309715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.417605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630671d-6bc9-4735-a746-2339330d1e70 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.476284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0eec7-f21a-43fb-a50c-eec2fb7e9ebe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.499993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:54.516512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.517432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.517746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.518051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.520052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.521627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.523496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 68/131] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:54 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:54.526186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.531181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.532095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.532496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.532787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.534103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.535665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.540332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 64/132] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:54 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:54.540332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.653287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5446e5a-514c-460d-9f81-32b8b279f82b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.655787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ba19-0212-4f33-8145-c1631b2c30eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.688760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:54.702158 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.703041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.703330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.703610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.705275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.707322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.710349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 65/133] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:54 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:54.711176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.712122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.713114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.713402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.713683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/a2bb088e25f642dba7a1e5a300b90359/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.717384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] RBAC: Authorizing `identity:create_grant(project_id=b58b1b03bb4947b4be69f8dc27be3cb4, user_id=a2bb088e25f642dba7a1e5a300b90359, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.719097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.762215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 69/134] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:29:54 2022] PUT /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/a2bb088e25f642dba7a1e5a300b90359/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:29:54.763815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:54.811418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:54.873999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4058074-4f27-4e64-a372-3334c3a080a3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:54.966234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:54.967490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:54.967791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:54.968079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] PATH_INFO: `/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:54.973094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] RBAC: Authorizing `identity:get_project(project_id=69ebd561e1bf46d5b4fc1e44252f986e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:54.975841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:54.978362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 66/135] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:54 2022] GET /identity/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:54.987307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:55.006112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3e377-c1ee-4e7b-be59-9cec5421cb17 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:55.064233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949f7f9-6859-4645-8231-492a5a6197fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:55.065122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949f7f9-6859-4645-8231-492a5a6197fd None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:55.065409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949f7f9-6859-4645-8231-492a5a6197fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:55.065692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949f7f9-6859-4645-8231-492a5a6197fd None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/8243f73b32484519bc2bed2bc9edc57e/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:55.067806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949f7f9-6859-4645-8231-492a5a6197fd None admin] RBAC: Authorizing `identity:create_grant(project_id=b58b1b03bb4947b4be69f8dc27be3cb4, user_id=8243f73b32484519bc2bed2bc9edc57e, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:55.070044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949f7f9-6859-4645-8231-492a5a6197fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:55.094552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 70/136] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:29:54 2022] PUT /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/8243f73b32484519bc2bed2bc9edc57e/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:29:55.176011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef968d-ae52-463f-99d2-6b041653283b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:55.205024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:55.226369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7af4-5ad6-4d3e-ad60-57f06980b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:55.227312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7af4-5ad6-4d3e-ad60-57f06980b00d None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:55.227603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7af4-5ad6-4d3e-ad60-57f06980b00d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:55.227943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7af4-5ad6-4d3e-ad60-57f06980b00d None admin] PATH_INFO: `/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e/users/a2bb088e25f642dba7a1e5a300b90359/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:55.230359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f7af4-5ad6-4d3e-ad60-57f06980b00d None admin] RBAC: Authorizing `identity:create_grant(project_id=69ebd561e1bf46d5b4fc1e44252f986e, user_id=a2bb088e25f642dba7a1e5a300b90359, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:55.232023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f7af4-5ad6-4d3e-ad60-57f06980b00d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:55.250547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 67/137] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:29:54 2022] PUT /identity/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e/users/a2bb088e25f642dba7a1e5a300b90359/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:29:56.716665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9583f0-5acc-4e8c-91d2-cc33eac4cffb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:56.717183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9583f0-5acc-4e8c-91d2-cc33eac4cffb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:56.717290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9583f0-5acc-4e8c-91d2-cc33eac4cffb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:56.718653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 71/138] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:56.724711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90fcb6-2626-4c03-bd3d-4dedb230097a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:56.725760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90fcb6-2626-4c03-bd3d-4dedb230097a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:56.725830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90fcb6-2626-4c03-bd3d-4dedb230097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:56.752674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1c90fcb6-2626-4c03-bd3d-4dedb230097a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:56.753910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c90fcb6-2626-4c03-bd3d-4dedb230097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:56.842639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 68/139] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:29:56.885353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9583f0-5acc-4e8c-91d2-cc33eac4cffb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:56.953102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:57.030851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768733a-5e54-42a5-8b47-d6d766d296fb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.031473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768733a-5e54-42a5-8b47-d6d766d296fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.031949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768733a-5e54-42a5-8b47-d6d766d296fb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.033718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 69/140] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:29:57.047160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.047872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.048400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.051645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9583f0-5acc-4e8c-91d2-cc33eac4cffb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:57.089782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:57.090967 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:57.126910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:57.127743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.128098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.128409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.153077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:57.162078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:57.163006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:29:57.163771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:57.164993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 72/141] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:57.172616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:57.270385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 70/142] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:57.332700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:57.388687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:57.409701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564015-8b8b-403b-840a-5ed549e2f7e2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:57.414411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:57.504744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:57.505710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.508417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.508417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.508417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:57.508844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:57.516334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:57.538386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 73/143] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:57.545553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:57.567429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397f6a-c0e8-47db-aa3b-39204d5e6612 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:57.650570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:57.651773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.652266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.652775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.657115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:57.658880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:57.660903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 71/144] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:57.667263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:57.667543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fab7b-ae1a-4509-a428-9b9daaee9366 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:57.723110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:57.724023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.724331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.724630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] PATH_INFO: `/v3/users/2d38e8d963004c2280138a3d42b00503` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.729116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] RBAC: Authorizing `identity:get_user(user_id=2d38e8d963004c2280138a3d42b00503)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:57.730614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:57.732332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 74/145] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:57 2022] GET /identity/v3/users/2d38e8d963004c2280138a3d42b00503 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:57.741021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:57.785051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:57.882633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d23a3-dc8f-42e3-af0d-1d3e2025905b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:57.914419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28bad3-bb34-4066-a2c5-67cc5c5957b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:57.924652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:57.931158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551dc49-9ded-44a6-aa56-77fc61a3bb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:57.932060 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551dc49-9ded-44a6-aa56-77fc61a3bb97 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.932373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551dc49-9ded-44a6-aa56-77fc61a3bb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.932657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551dc49-9ded-44a6-aa56-77fc61a3bb97 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.936066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551dc49-9ded-44a6-aa56-77fc61a3bb97 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:57.937561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551dc49-9ded-44a6-aa56-77fc61a3bb97 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:57.944422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 75/146] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:57 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:29:57.949444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1375-e069-4765-ac84-caa2d985bfbc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.949740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1375-e069-4765-ac84-caa2d985bfbc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.950018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1375-e069-4765-ac84-caa2d985bfbc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.951775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 76/147] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:57.954737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f1375-e069-4765-ac84-caa2d985bfbc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:57.986411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ce620-46a1-4b09-98ba-8607eac0ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:57.987163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ce620-46a1-4b09-98ba-8607eac0ec5d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:57.987454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ce620-46a1-4b09-98ba-8607eac0ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:57.987736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ce620-46a1-4b09-98ba-8607eac0ec5d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:57.988549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ce620-46a1-4b09-98ba-8607eac0ec5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:57.990050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ce620-46a1-4b09-98ba-8607eac0ec5d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:58.027112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 72/148] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:29:57 2022] POST /identity/v3/users => generated 283 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:29:58.028234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a260-e3a4-44bb-bc94-9cc67aa0c039 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.029205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a260-e3a4-44bb-bc94-9cc67aa0c039 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.029500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14a260-e3a4-44bb-bc94-9cc67aa0c039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.076192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf14a260-e3a4-44bb-bc94-9cc67aa0c039 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:58.077478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf14a260-e3a4-44bb-bc94-9cc67aa0c039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:58.193357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:58.214258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f1375-e069-4765-ac84-caa2d985bfbc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:58.237950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 73/149] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:58.239239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347db12-4829-4fd7-8717-7cdb20c42f91 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.239539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347db12-4829-4fd7-8717-7cdb20c42f91 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.239818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347db12-4829-4fd7-8717-7cdb20c42f91 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.245170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 74/150] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:58.247357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947efee-18b6-441c-9dd8-9dec1f691d1a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.247649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947efee-18b6-441c-9dd8-9dec1f691d1a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.247927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947efee-18b6-441c-9dd8-9dec1f691d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.286139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0947efee-18b6-441c-9dd8-9dec1f691d1a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:58.290217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0947efee-18b6-441c-9dd8-9dec1f691d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:58.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:58.297691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.297972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.298246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/2d38e8d963004c2280138a3d42b00503/roles/c71df34c0f1c4a2a8f0bff66914681f7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.300453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] RBAC: Authorizing `identity:create_grant(project_id=b58b1b03bb4947b4be69f8dc27be3cb4, user_id=2d38e8d963004c2280138a3d42b00503, role_id=c71df34c0f1c4a2a8f0bff66914681f7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:58.301937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:58.328450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 77/151] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:29:57 2022] PUT /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/users/2d38e8d963004c2280138a3d42b00503/roles/c71df34c0f1c4a2a8f0bff66914681f7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:29:58.329115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:58.407117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 75/152] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:58.414473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3c389-0e68-400b-9631-370f333fe2ad None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.414878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3c389-0e68-400b-9631-370f333fe2ad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.415031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3c389-0e68-400b-9631-370f333fe2ad None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.416669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 76/153] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:58.422296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.422580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.422857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.450725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:58.451982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:58.476632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:58.482100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45ec61-2c43-489b-a4b2-216506ae06ac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:58.526478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:58.527400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.527691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.527971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.541308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:58.542357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 77/154] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:58.543044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:58.551473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:58.554403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:58.556363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 78/155] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:29:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:58.568368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:58.580220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:58.671568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0728-850d-4092-9637-4e1c9c231fb5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:58.681680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d42f8a-f205-476d-a2d6-cdd79066d608 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:58.720324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:58.721182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.721472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.721912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.726446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:58.727137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.727424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.727704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.728494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:58.728970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:58.730682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:58.734582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:58.744258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:58.745581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:29:58.746562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:29:58.747269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 79/156] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:29:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:58.748511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:58.748911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 78/157] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:29:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:58.749884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:58.772081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:58.863349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:58.883083 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77ab69-b4f7-4028-a50a-28bd5d586e01 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:58.903449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c53e-2950-4c96-a583-6091026af8a3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:58.927912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:58.929712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.929995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.930275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:58.935851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:58.935851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:58.935851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 79/158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:58 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:58.936674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:58.976942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:58.978398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:58.979091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:58.979600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.000634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:29:59.010111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:29:59.011031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:29:59.011879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 80/159] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:29:59.012578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:59.034423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:59.063394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40707b5-f301-4a72-80d2-6a8b5f03479d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.107900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:59.108776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.109400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.109693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.110726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:59.111040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:59.112635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f5ece-d4ac-4ee3-b1d5-a3db1ce9dbe4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:59.123547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 80/160] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:29:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:59.125377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302924c-cb63-44ef-a175-418cdb12d375 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.125670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302924c-cb63-44ef-a175-418cdb12d375 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.125952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302924c-cb63-44ef-a175-418cdb12d375 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.127230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 81/161] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:29:59.127953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302924c-cb63-44ef-a175-418cdb12d375 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:59.169024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2489759-229b-4e10-bd84-91a19cb38c05 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.215219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d26fe-6663-41db-9124-676227b2f28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:59.215884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d26fe-6663-41db-9124-676227b2f28d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.216175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d26fe-6663-41db-9124-676227b2f28d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.216485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d26fe-6663-41db-9124-676227b2f28d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.232903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:59.263581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302924c-cb63-44ef-a175-418cdb12d375 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.310613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:59.311801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.312085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.312391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.313160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:29:59.313469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:59.314902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:59.321914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d26fe-6663-41db-9124-676227b2f28d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:59.323901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d26fe-6663-41db-9124-676227b2f28d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:59.327233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 82/162] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:59.330459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:59.341090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:59.462621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 81/163] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:29:59.463953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.464251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.465193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80deb-4580-445e-aad5-35945d9f706b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.467818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.489256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:29:59.491298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:29:59.508342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:59.509415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.510103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.510403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] PATH_INFO: `/v3/users/8243f73b32484519bc2bed2bc9edc57e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.511951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] RBAC: Authorizing `identity:get_user(user_id=8243f73b32484519bc2bed2bc9edc57e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:59.513676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:59.515710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 83/164] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:59 2022] GET /identity/v3/users/8243f73b32484519bc2bed2bc9edc57e => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:59.516528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:59.538007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:59.608413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:29:59.610946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 82/165] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:29:59.612110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:59.638289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f7903-6f12-4572-8f9d-543cfb026be7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.690544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:59.691435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.691717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.691995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:29:59.693455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:29:59.695502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:29:59.697636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 84/166] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:29:59 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:29:59.698215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:29:59.835818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030af01e-694f-42af-9d0f-026a8fe98967 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.928209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9a78a-2e3b-45f7-a55b-5da2de4c2010 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:29:59.985089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:29:59.985781 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:29:59.986077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:29:59.986367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:00.011118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:00.012491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:00.014863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:00.015460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:00.015620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:00.018351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:00.025278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:00.027158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 83/167] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:29:59 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:00.027948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:00.092076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:00.168536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:00.170501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:00.218744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c451d-7968-41b1-a55f-aa7e5876c60b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:00.292773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:00.293637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:00.293922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:00.294197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:00.312617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:00.326353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:00.327291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:00.328073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 84/168] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:00.328833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:00.391488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 85/169] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:00.392969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:00.490928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:00.580669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:00.580893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731f499-6815-4931-822e-3e98e64f3917 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:00.602370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cb5ed-8d09-40a6-8d5f-b2f820713dbb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:00.619817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:00.620576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:00.620863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:00.621144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:00.622036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] RBAC: Adding query filter params (role.id=9508980101c44da48df022abc45e1e60, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=a2bb088e25f642dba7a1e5a300b90359) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:00.622355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:00.623907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:00.630363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 86/170] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:00 2022] GET /identity/v3/role_assignments?user.id=a2bb088e25f642dba7a1e5a300b90359&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=9508980101c44da48df022abc45e1e60 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:00.631896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:00.725302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:00.726004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:00.726297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:00.726587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:00.759752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937368-6dc2-4c01-a2c5-1eba86f1c6f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:00.773868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:00.835191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:00.836021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:00.836357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:00.836673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:00.837554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:00.837888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:00.839308 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:00.858073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 87/171] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:00.858944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:00.866874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:00.874061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.039633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ae63d-d73d-4527-b4f9-bdf10c5bb069 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:01.088706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:01.102677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 85/172] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 774 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:01.103562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:01.119933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:01.120861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.121154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.121437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.122738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:01.124236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.125984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 88/173] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:00 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:01.127295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:01.214549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:01.346473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9e83f-5707-408c-b18d-214ad03e5dd3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:01.360138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4edfcc-1412-4721-a185-c0ca2ebfc31b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:01.417911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:01.418798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.419086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.419367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] PATH_INFO: `/v3/users/a2bb088e25f642dba7a1e5a300b90359` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.421022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] RBAC: Authorizing `identity:get_user(user_id=a2bb088e25f642dba7a1e5a300b90359)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:01.422492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.424133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 86/174] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:01 2022] GET /identity/v3/users/a2bb088e25f642dba7a1e5a300b90359 => generated 275 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:01.424938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:01.431983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:01.433139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.433568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.433989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.441141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:01.442829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.445829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 89/175] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:01 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:01.446780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:01.527747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:01.598745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca5f3-e1bc-41ee-92c2-9eecb021b94f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:01.622021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848497ec-1cc7-4480-917e-45a8ed932935 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:01.638052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:01.677629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:01.679017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.679313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.679591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.680469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=8243f73b32484519bc2bed2bc9edc57e) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:01.680787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:01.682258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.697141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db20f3-6120-4c62-bf13-dd09af249b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:01.697343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 87/176] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:01 2022] GET /identity/v3/role_assignments?user.id=8243f73b32484519bc2bed2bc9edc57e&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:01.697869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db20f3-6120-4c62-bf13-dd09af249b40 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.698142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db20f3-6120-4c62-bf13-dd09af249b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.698263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:01.698416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db20f3-6120-4c62-bf13-dd09af249b40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.875111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db20f3-6120-4c62-bf13-dd09af249b40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:01.876547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db20f3-6120-4c62-bf13-dd09af249b40 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.896143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92feb25-d1e6-4701-8ffa-8675636e1838 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:01.955288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:01.969453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:01.970607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.971093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.971593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.972697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=a2bb088e25f642dba7a1e5a300b90359) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:01.973294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:01.975187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa56af6-83f5-4bcc-91e5-7b35c3d67b4a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:01.982489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 88/177] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:01 2022] GET /identity/v3/role_assignments?user.id=a2bb088e25f642dba7a1e5a300b90359&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=ee34d08904734afdb4939fd27d04b760 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:01.984388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edf36c-d8aa-4b3a-8cc3-e77fdafdde71 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.984872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edf36c-d8aa-4b3a-8cc3-e77fdafdde71 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.985344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edf36c-d8aa-4b3a-8cc3-e77fdafdde71 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.986847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 89/178] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:01.988415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91685e-114a-477c-b924-6034a882a3c3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:01.988948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91685e-114a-477c-b924-6034a882a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:01.989421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91685e-114a-477c-b924-6034a882a3c3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:01.991112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 90/179] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:02.002172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.002791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.003360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.041640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:02.043330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:02.053872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 90/180] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:02.055014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.055310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.055591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.075136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:02.097512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:02.098766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:02.171927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 91/181] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:02.172773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.267512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 91/182] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:02.267819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.346614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6e229-9877-43b1-97ec-cf3dabdb6453 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:02.368820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:02.391798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:02.392517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.392793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.393079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] PATH_INFO: `/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.394387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] RBAC: Authorizing `identity:get_project(project_id=69ebd561e1bf46d5b4fc1e44252f986e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:02.395892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:02.399491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 92/183] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/projects/69ebd561e1bf46d5b4fc1e44252f986e => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:02.400107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.445075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f0f9-8fc6-4a7a-b0e9-f29c2d8c81c8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:02.486029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:02.486887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.487126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.487425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.489670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:02.500584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:02.509030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:02.510020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:02.510819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 92/184] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:02.511470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.548638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e08d-8ed7-4ae3-9f3b-52219cf347a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:02.591756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:02.593087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.593594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.594030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.607056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:02.616286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:02.617253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:02.618054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 93/185] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:02.618710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.642678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb541790-4f44-46b9-a8ad-1f7fd74f7f42 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:02.673246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:02.694351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:02.695559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.696039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.696606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.698328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=69ebd561e1bf46d5b4fc1e44252f986e, user.id=a2bb088e25f642dba7a1e5a300b90359) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:02.698982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:02.702553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029f090-55fe-42a0-8b0f-9a9eac002b50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:02.724513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 93/186] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/role_assignments?user.id=a2bb088e25f642dba7a1e5a300b90359&scope.project.id=69ebd561e1bf46d5b4fc1e44252f986e&role.id=ee34d08904734afdb4939fd27d04b760 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:02.724513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94862d-100f-4e08-ad66-5a38c27a97a0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.724513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94862d-100f-4e08-ad66-5a38c27a97a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.724513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94862d-100f-4e08-ad66-5a38c27a97a0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.724513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 94/187] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:02.724513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94862d-100f-4e08-ad66-5a38c27a97a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.748742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e539-91ec-4bb4-b0b9-29ee4adddb6d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:02.795591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:02.796271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.796576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.796862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.797762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:02.798077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:02.799668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a671a-ca55-42e3-b96e-d49e7e18fd8e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:02.801815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:02.816380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 94/188] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:02.816380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb546-dca2-4399-8701-0a8e5641e3a3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.816380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb546-dca2-4399-8701-0a8e5641e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.816380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb546-dca2-4399-8701-0a8e5641e3a3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.816380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 95/189] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:02.817108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ac892-870e-4754-a169-56c65197954a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.817108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ac892-870e-4754-a169-56c65197954a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.817647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ac892-870e-4754-a169-56c65197954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.841576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a15ac892-870e-4754-a169-56c65197954a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:02.842829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a15ac892-870e-4754-a169-56c65197954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:02.873964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94862d-100f-4e08-ad66-5a38c27a97a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:02.919253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:02.919941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.920224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.920533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.921300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:02.921606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:02.924440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:02.933904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 95/190] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:02.934614 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:02.939148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 96/191] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:02.940605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:02.941017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:02.941250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:02.964795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:02.965892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:03.004976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.062045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516c01-2250-48a0-8538-3d3218445afa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.062584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 97/192] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:03.063431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.089636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.106684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf07490-494b-407b-b573-b451315124ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.107568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07490-494b-407b-b573-b451315124ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.107845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07490-494b-407b-b573-b451315124ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.108113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07490-494b-407b-b573-b451315124ca None admin] PATH_INFO: `/v3/users/2d38e8d963004c2280138a3d42b00503` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.109384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf07490-494b-407b-b573-b451315124ca None admin] RBAC: Authorizing `identity:get_user(user_id=2d38e8d963004c2280138a3d42b00503)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.110864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf07490-494b-407b-b573-b451315124ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.112577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 96/193] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:02 2022] GET /identity/v3/users/2d38e8d963004c2280138a3d42b00503 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:03.112577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf07490-494b-407b-b573-b451315124ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.193760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999571d-3868-4c6f-ae4d-81436350cbef None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.231856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf07490-494b-407b-b573-b451315124ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.238355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.239258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.239822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.239822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] PATH_INFO: `/v3/users/e72143a3716d4ef3bb8ad6e54353c383` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.254360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] RBAC: Authorizing `identity:get_user(user_id=e72143a3716d4ef3bb8ad6e54353c383)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.255865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.257876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 98/194] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/users/e72143a3716d4ef3bb8ad6e54353c383 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:03.258674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.268679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.269606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.269922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.270227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.272058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.273558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.275176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 97/195] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:03.276052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.286371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.338454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.366937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaa489-b4d9-478a-b7aa-55225ebeab39 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.388163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256e542-6836-402a-8f41-4fc9a674b122 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.403594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.404446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.404729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.405006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.415561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:03.423797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:03.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 99/196] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:03.431097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.446962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.448102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.448652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.449176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.523419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.538792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b1e4f-4e56-4f91-8642-da2f381c8757 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.544858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.546355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.577316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.577974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.578258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.578535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.610506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.658585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.658793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 98/197] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:03.659577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.660164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.754417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.769832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7238a11-2040-4f2c-b96a-7eb4d647f786 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.771793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 100/198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:03.772704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.820174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.820860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.821140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.821472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.822214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.823742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.845641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 99/199] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:03 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:03.850003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:03.888364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:03.923927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c41ab0-1bfb-4a2e-9b03-77a54a39c385 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:03.966400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:03.967094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:03.967378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:03.967657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:03.968487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:03.968785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:03.970221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:03.979591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 101/200] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:03.980496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:04.015791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1ed7e-58e7-44ac-bb81-35b0fe0fce50 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:04.041992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:04.061323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:04.062458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:04.064131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:04.064131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:04.066236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:04.069445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:04.071643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 100/201] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:04.072339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:04.104352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781e46-92ed-4aa0-a2d7-dfcd4fd8622d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:04.150468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:04.151355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:04.151637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:04.151913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:04.154897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:04.154897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:04.156833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 102/202] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:03 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:04.157545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:04.221253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:04.261504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff6a41-1cf2-458f-8f5f-ac7e2a30b464 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:04.292029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde802-be09-4451-84b0-e39c856b5020 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:04.306231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:04.307346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:04.307642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:04.308100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] PATH_INFO: `/v3/users/e72143a3716d4ef3bb8ad6e54353c383` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:04.309881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] RBAC: Authorizing `identity:get_user(user_id=e72143a3716d4ef3bb8ad6e54353c383)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:04.311671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:04.311783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:04.314337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 101/203] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:04 2022] GET /identity/v3/users/e72143a3716d4ef3bb8ad6e54353c383 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:04.315138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:04.337395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:04.337989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:04.338278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:04.338558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:04.339460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] RBAC: Adding query filter params (role.id=c71df34c0f1c4a2a8f0bff66914681f7, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4, user.id=2d38e8d963004c2280138a3d42b00503) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:04.339777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:04.341312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:04.347854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 103/204] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:04 2022] GET /identity/v3/role_assignments?user.id=2d38e8d963004c2280138a3d42b00503&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=c71df34c0f1c4a2a8f0bff66914681f7 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:04.361112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:04.498533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c15210-4313-449a-b1f8-a861606fb89d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:04.581146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:04.581832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:04.582121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:04.582403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:04.583284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=e72143a3716d4ef3bb8ad6e54353c383) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:04.583580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:04.585687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:04.594300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e31f7-2a13-419b-b444-1a9afa030f7a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:04.602436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 102/205] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:04 2022] GET /identity/v3/role_assignments?user.id=e72143a3716d4ef3bb8ad6e54353c383&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:04.638170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:04.638877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:04.639169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:04.639673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:04.703350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:04.728674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:04.730228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:04.869719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 104/206] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:04.875673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:04.901117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:05.051111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0fc86-8dbd-4f56-9552-d35b9eff477b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:05.087797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a7fcb-5623-40bb-85ad-5257fc543b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:05.088704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a7fcb-5623-40bb-85ad-5257fc543b0c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:05.088984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a7fcb-5623-40bb-85ad-5257fc543b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:05.089260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a7fcb-5623-40bb-85ad-5257fc543b0c None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:05.090515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a7fcb-5623-40bb-85ad-5257fc543b0c None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:05.092020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a7fcb-5623-40bb-85ad-5257fc543b0c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:05.098599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 103/207] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:04 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:05.100581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:05.194071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:05.214938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361e9a-178a-4884-bcb8-b28fa56461fc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:05.254831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:05.255517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:05.255839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:05.256121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:05.257177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] RBAC: Adding query filter params (role.id=9508980101c44da48df022abc45e1e60, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=e72143a3716d4ef3bb8ad6e54353c383) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:05.257493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:05.258966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d20212-b2d7-44b3-a631-cacdb537bab5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:05.265526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 105/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:05 2022] GET /identity/v3/role_assignments?user.id=e72143a3716d4ef3bb8ad6e54353c383&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=9508980101c44da48df022abc45e1e60 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:06.382046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa88a9c-9341-40b3-98f8-9213ecea92a2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.382046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa88a9c-9341-40b3-98f8-9213ecea92a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.382046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa88a9c-9341-40b3-98f8-9213ecea92a2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.383588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 104/209] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:06.389772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996e55b-f850-4656-a34c-7d82ce65f198 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.390308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996e55b-f850-4656-a34c-7d82ce65f198 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.390308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996e55b-f850-4656-a34c-7d82ce65f198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.408448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7996e55b-f850-4656-a34c-7d82ce65f198 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:06.409652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7996e55b-f850-4656-a34c-7d82ce65f198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:06.485076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 106/210] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:06.510935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa88a9c-9341-40b3-98f8-9213ecea92a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:06.580741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f441db1-3f48-4b6c-a363-906abd15ad49 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.581032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f441db1-3f48-4b6c-a363-906abd15ad49 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.581305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f441db1-3f48-4b6c-a363-906abd15ad49 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.582777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 107/211] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:06.588749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.589036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.589318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.608949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:06.610103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:06.611702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:06.623682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa88a9c-9341-40b3-98f8-9213ecea92a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:06.668231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:06.669000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.669286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.669566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.683823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:06.693659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:06.694790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:06.695604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:06.700320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 105/212] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:06.701259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:06.716434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 108/213] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:06.745035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:06.746221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:06.833266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cf9da-8593-4883-98a1-b2eb31220cbf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:06.859910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f665e-ed52-4935-991d-5fcf6f7c6367 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:06.872684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:06.873665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.874173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.874748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.875784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:06.876291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:06.878077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:06.888519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 106/214] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:06.893924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:06.897486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:06.903132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:06.903945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:06.904223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:06.904525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:06.905757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:06.907373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:06.914099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 109/215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:06.921002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:06.999521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:07.027163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9c99-f797-4c40-a6e4-a4305ff7a0af None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.033817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9127273-db19-4f5d-9afb-c9e3000e96ed None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.067483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.069382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.069849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.070305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] PATH_INFO: `/v3/users/5360b748742a49cc816f9330b9c5a340` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.076951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f311c4d-414b-4c12-986c-4257591634c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.077744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f311c4d-414b-4c12-986c-4257591634c2 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.078095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f311c4d-414b-4c12-986c-4257591634c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.078437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f311c4d-414b-4c12-986c-4257591634c2 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.079232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f311c4d-414b-4c12-986c-4257591634c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.082214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f311c4d-414b-4c12-986c-4257591634c2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.088075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] RBAC: Authorizing `identity:get_user(user_id=5360b748742a49cc816f9330b9c5a340)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.089885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.092165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 107/216] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:06 2022] GET /identity/v3/users/5360b748742a49cc816f9330b9c5a340 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:07.100719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.125965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 110/217] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:06 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:30:07.127191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d2766-dc2c-49a2-91d2-a71f752fb678 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.127674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d2766-dc2c-49a2-91d2-a71f752fb678 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.128004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d2766-dc2c-49a2-91d2-a71f752fb678 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.129814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 111/218] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:07.143464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.143748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.144233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.174926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:07.176612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:07.187734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:07.256136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e557a1-634d-42cf-a4cc-f637aa98b2ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.269413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 112/219] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:07.295037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.296649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.297549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.297834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.299031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.379141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:07.396123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.397745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.444707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14703e6-cadc-4240-95eb-0191cea68c38 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.476639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:07.484658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.485573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.485870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.486136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.501433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:07.514410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:07.515364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:07.516204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:07.516860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 113/220] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:07.528732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.541672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 108/221] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:30:07.549232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.673916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a50fe-25ba-4b8e-8dad-57f5a822b349 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.684087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f102e-2b5e-449c-b4e2-d80707a87f49 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.697974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:07.724175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.726403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.726687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.726971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.727743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:07.728049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.729573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.734406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.735282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.735562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.735838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.737722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.739257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.742040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 109/222] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:07.748598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.750735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 114/223] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:07.757419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.790736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:07.892445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d90169-eea8-4c40-9c52-041027865e8c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.907363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9453f-67ca-49f7-bc2f-99d5db318acd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:07.943499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.944388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.944678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.945017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.946764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=5360b748742a49cc816f9330b9c5a340) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:07.947095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.954605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286177ab-4f8c-42e0-9537-bece16e7c9f9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.965391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40513c03-32f1-4bc9-b25c-4888219c2c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:07.966313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 110/224] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/role_assignments?user.id=5360b748742a49cc816f9330b9c5a340&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=ee34d08904734afdb4939fd27d04b760 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:07.967193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513c03-32f1-4bc9-b25c-4888219c2c53 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.967732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513c03-32f1-4bc9-b25c-4888219c2c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.968286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513c03-32f1-4bc9-b25c-4888219c2c53 None admin] PATH_INFO: `/v3/users/e72143a3716d4ef3bb8ad6e54353c383` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.968505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30499105-e3cd-41fc-a608-f629a7faa3f2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.968759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30499105-e3cd-41fc-a608-f629a7faa3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.969044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30499105-e3cd-41fc-a608-f629a7faa3f2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:07.970636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40513c03-32f1-4bc9-b25c-4888219c2c53 None admin] RBAC: Authorizing `identity:get_user(user_id=e72143a3716d4ef3bb8ad6e54353c383)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:07.972657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40513c03-32f1-4bc9-b25c-4888219c2c53 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:07.975372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 115/225] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/users/e72143a3716d4ef3bb8ad6e54353c383 => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:07.984396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 111/226] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:07.986551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:07.988126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30499105-e3cd-41fc-a608-f629a7faa3f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:07.991351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:07.991899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:08.022031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:08.023754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:08.143243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:08.146814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:08.162304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30499105-e3cd-41fc-a608-f629a7faa3f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:08.198442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 116/227] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:08.219485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:08.220871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:08.221556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:08.222148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:08.224247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:08.227799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:08.231100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 112/228] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:07 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:08.237991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:08.267612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:08.402870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa292b-4645-4b7c-9752-572c0f1723ec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:08.465100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:08.466618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:08.467087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:08.469188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9/users/e72143a3716d4ef3bb8ad6e54353c383/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:08.474227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44454738f6541b6aa730195de6989a9, user_id=e72143a3716d4ef3bb8ad6e54353c383, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:08.477734 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:08.513887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 117/229] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:30:08 2022] PUT /identity/v3/projects/d44454738f6541b6aa730195de6989a9/users/e72143a3716d4ef3bb8ad6e54353c383/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:08.523645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5609d8-7da9-4461-b1ce-32b33bdc54cf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:08.587169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:08.588014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:08.588297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:08.588602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:08.610600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:08.625039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:08.626031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:08.626831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:08.627942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 113/230] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:08.634230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:08.706773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:08.780594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a364af-eb6a-4395-8fd4-8aa1dd015cd0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:08.822782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:08.823810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:08.824290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:08.824809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:08.825793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:08.826311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:08.828201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:08.839056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 118/231] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:08.844824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:08.876774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:08.975077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81cbb6-37a6-479f-87db-08e812fdcfda None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:09.015578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:09.016666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:09.017051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:09.017501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] PATH_INFO: `/v3/users/e72143a3716d4ef3bb8ad6e54353c383` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:09.025086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] RBAC: Authorizing `identity:get_user(user_id=e72143a3716d4ef3bb8ad6e54353c383)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:09.026808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:09.028860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 114/232] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:08 2022] GET /identity/v3/users/e72143a3716d4ef3bb8ad6e54353c383 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:09.035256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:09.194390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c577370-31df-4074-ad7d-faf2eb412d25 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:09.201934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:09.237614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeeac06-8595-4b2a-b361-f96ed67f409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:09.238549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeac06-8595-4b2a-b361-f96ed67f409f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:09.238835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeac06-8595-4b2a-b361-f96ed67f409f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:09.239110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeac06-8595-4b2a-b361-f96ed67f409f None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:09.241120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeeac06-8595-4b2a-b361-f96ed67f409f None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:09.242773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeeac06-8595-4b2a-b361-f96ed67f409f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:09.244912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 119/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:09 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:09.250367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:09.392997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:09.405638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cb8fb-508b-439f-995a-926c44184da8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:09.471685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132851da-0d19-4172-b32b-757d4dcdd791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:09.472652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132851da-0d19-4172-b32b-757d4dcdd791 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:09.472938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132851da-0d19-4172-b32b-757d4dcdd791 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:09.473218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132851da-0d19-4172-b32b-757d4dcdd791 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9/users/e72143a3716d4ef3bb8ad6e54353c383/roles/9508980101c44da48df022abc45e1e60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:09.476464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132851da-0d19-4172-b32b-757d4dcdd791 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44454738f6541b6aa730195de6989a9, user_id=e72143a3716d4ef3bb8ad6e54353c383, role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:09.477952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132851da-0d19-4172-b32b-757d4dcdd791 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:09.502926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 115/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:30:09 2022] PUT /identity/v3/projects/d44454738f6541b6aa730195de6989a9/users/e72143a3716d4ef3bb8ad6e54353c383/roles/9508980101c44da48df022abc45e1e60 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:10.273506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667657-50e8-4b11-97f7-f99f6a6e9f3f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.274409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667657-50e8-4b11-97f7-f99f6a6e9f3f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.274843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667657-50e8-4b11-97f7-f99f6a6e9f3f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.276835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 120/235] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:10.299292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff1ff6-8e8a-42f1-b655-97adb25856f6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.300181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff1ff6-8e8a-42f1-b655-97adb25856f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.300635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff1ff6-8e8a-42f1-b655-97adb25856f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.320112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-01ff1ff6-8e8a-42f1-b655-97adb25856f6 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:10.325855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01ff1ff6-8e8a-42f1-b655-97adb25856f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:10.434955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 116/236] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:10.460325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d88a09-6600-482d-9a32-7785aecba8de None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.460640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d88a09-6600-482d-9a32-7785aecba8de None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.460932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d88a09-6600-482d-9a32-7785aecba8de None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.462333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 121/237] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:10.468289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.468710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.469067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.486773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d88a09-6600-482d-9a32-7785aecba8de None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:10.494754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:10.495962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:10.633639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 117/238] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:10.684184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:10.697332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:10.706910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d88a09-6600-482d-9a32-7785aecba8de None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:10.785772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:10.786586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.786934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.787215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.805799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:10.821457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:10.822425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:10.823216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:10.823823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 122/239] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:10.837134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:10.900783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8b0e1-d7d5-4f99-87bc-33b76059f3d9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:10.900968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:10.965081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796d372-14bb-491f-b3c2-8e61a7ea769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:10.965895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d372-14bb-491f-b3c2-8e61a7ea769d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.966180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d372-14bb-491f-b3c2-8e61a7ea769d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.966456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d372-14bb-491f-b3c2-8e61a7ea769d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.967838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796d372-14bb-491f-b3c2-8e61a7ea769d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:10.969357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796d372-14bb-491f-b3c2-8e61a7ea769d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:10.971217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 118/240] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:10.976346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f1304-08ab-4add-b5fd-02f7bc349d32 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:10.976346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f1304-08ab-4add-b5fd-02f7bc349d32 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:10.976346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f1304-08ab-4add-b5fd-02f7bc349d32 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:10.977486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 119/241] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:10.978215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f1304-08ab-4add-b5fd-02f7bc349d32 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:11.018688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c92988-7ec0-43c1-85c7-a529dbd12f6c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:11.083099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:11.084277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.084801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.085046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:11.085797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.087856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:11.088862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:11.092082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0de4c-3777-4c22-84d6-3abbcc9cb170 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:11.114189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 123/242] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:11.117899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.117899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.118144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.124337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f1304-08ab-4add-b5fd-02f7bc349d32 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:11.143332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:11.145001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:11.178667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:11.179916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.180462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.180941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.182295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:11.185240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:11.221127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 120/243] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:10 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:11.222828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:11.232194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 124/244] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:11.262784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:11.331056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:11.434103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33d5a0-da3e-4d7e-8412-9e83f8f27b00 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:11.468489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:11.480262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:11.481188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.481472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.481750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] PATH_INFO: `/v3/users/99bd9c42a52a4584a42b89bbf316e394` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.502291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] RBAC: Authorizing `identity:get_user(user_id=99bd9c42a52a4584a42b89bbf316e394)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:11.504712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:11.506905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 121/245] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:11 2022] GET /identity/v3/users/99bd9c42a52a4584a42b89bbf316e394 => generated 297 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:11.513324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:11.533115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bbbb4-54fc-4135-8b62-9bea6f30411e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:11.593506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:11.594667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.595524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.596996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.616571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:11.624705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:11.625650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:11.626446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 125/246] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:11.634431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:11.685570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04626f0-4910-460d-a4fb-c38d41f9a2c6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:11.743626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:11.744417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.744708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.744987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.767469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:11.812588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab8d4d-f4b8-4e00-8109-9c72083d80f6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:11.838948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:11.859534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:11.861321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:11.879464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:11.880191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.881061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.881951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.882520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:11.882699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:11.884951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d560af-370d-46db-a650-41eea61666f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:11.898178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 126/247] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:11.899647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a66a-457c-4dea-92b9-cd2cfdb4be12 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.899933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a66a-457c-4dea-92b9-cd2cfdb4be12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.900398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5a66a-457c-4dea-92b9-cd2cfdb4be12 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.902246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 127/248] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:11.911353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:11.912366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:11.912763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:11.938592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:11.940268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:12.028539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 122/249] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:12.029413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.053214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 128/250] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:12.054850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.179873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:12.209648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:12.212999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95c6f8-6f5c-4a11-9814-de628a9235c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:12.246684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adf6c7-52e0-46e9-92cb-c7d7d17c4f26 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:12.279074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:12.279987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.280270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.280578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] PATH_INFO: `/v3/users/5360b748742a49cc816f9330b9c5a340` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.283575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] RBAC: Authorizing `identity:get_user(user_id=5360b748742a49cc816f9330b9c5a340)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:12.285175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:12.287026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 123/251] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:12 2022] GET /identity/v3/users/5360b748742a49cc816f9330b9c5a340 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:12.287996 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.330487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90325091-eeef-4686-9306-8639158d30de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:12.331369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90325091-eeef-4686-9306-8639158d30de None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.331912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90325091-eeef-4686-9306-8639158d30de None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.331912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90325091-eeef-4686-9306-8639158d30de None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.337707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90325091-eeef-4686-9306-8639158d30de None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:12.339205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90325091-eeef-4686-9306-8639158d30de None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:12.341689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 129/252] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:12 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:12.343880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90325091-eeef-4686-9306-8639158d30de None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.488541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90325091-eeef-4686-9306-8639158d30de None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:12.491026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28ed02-542e-42e8-b16e-fb971861555f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:12.528193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:12.529610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.530081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.530504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.532184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:12.533174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459d3b8-b5ee-4bd4-96b3-c46af39b7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:12.533884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:12.534618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d3b8-b5ee-4bd4-96b3-c46af39b7dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.535140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d3b8-b5ee-4bd4-96b3-c46af39b7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.535639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d3b8-b5ee-4bd4-96b3-c46af39b7dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.535815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 130/253] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:12 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:12.537490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.550213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:12.558378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459d3b8-b5ee-4bd4-96b3-c46af39b7dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7459d3b8-b5ee-4bd4-96b3-c46af39b7dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:12.573153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:12.574815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:12.588969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 124/254] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:12.591841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb4f50-8450-411e-bc16-d70c571c5ed5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.592770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb4f50-8450-411e-bc16-d70c571c5ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.593542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb4f50-8450-411e-bc16-d70c571c5ed5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.596814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 125/255] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:12 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:12.602847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb4f50-8450-411e-bc16-d70c571c5ed5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.635901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:12.676013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7229ef6-cd4b-4e82-bb7a-b15fe7686c04 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:12.723767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:12.726303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.726772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.727221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.728224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] RBAC: Adding query filter params (role.id=c71df34c0f1c4a2a8f0bff66914681f7, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=99bd9c42a52a4584a42b89bbf316e394) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:12.728705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:12.730481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:12.737718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 131/256] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:12 2022] GET /identity/v3/role_assignments?user.id=99bd9c42a52a4584a42b89bbf316e394&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=c71df34c0f1c4a2a8f0bff66914681f7 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:12.738888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:12.796339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb4f50-8450-411e-bc16-d70c571c5ed5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:12.864222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f45884-ae60-4b54-8f89-c6fdb64bf0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:12.866994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f45884-ae60-4b54-8f89-c6fdb64bf0c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.867310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f45884-ae60-4b54-8f89-c6fdb64bf0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.867594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f45884-ae60-4b54-8f89-c6fdb64bf0c3 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9/users/5360b748742a49cc816f9330b9c5a340/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.876459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f45884-ae60-4b54-8f89-c6fdb64bf0c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44454738f6541b6aa730195de6989a9, user_id=5360b748742a49cc816f9330b9c5a340, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:12.879389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f45884-ae60-4b54-8f89-c6fdb64bf0c3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:12.931676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 126/257] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:30:12 2022] PUT /identity/v3/projects/d44454738f6541b6aa730195de6989a9/users/5360b748742a49cc816f9330b9c5a340/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:12.938014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:12.938493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:12.938782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:12.961251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:12.976877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:12.979186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:13.071859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d6a0f-ec8f-4631-b31e-78df459fb4d5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:13.076931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 127/258] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:13.107209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:13.121537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:13.166014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:13.166712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:13.166998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:13.167277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:13.168046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:13.168386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:13.169801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:13.191282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 132/259] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:13.197723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:13.314860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7081c6-7d4a-4b16-b7c5-da5e948a3caf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:13.401279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:13.402043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:13.402446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:13.402728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:13.422213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:13.430840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:13.432645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:13.434008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 128/260] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:13.438936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:13.477260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4371611-e066-4678-8ca1-d4e0352dc4dc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:13.514289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:13.515179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:13.515458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:13.515732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] PATH_INFO: `/v3/users/e72143a3716d4ef3bb8ad6e54353c383` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:13.517108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] RBAC: Authorizing `identity:get_user(user_id=e72143a3716d4ef3bb8ad6e54353c383)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:13.518608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:13.520678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 133/261] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:13 2022] GET /identity/v3/users/e72143a3716d4ef3bb8ad6e54353c383 => generated 283 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:13.529817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:13.537874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:13.539827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:13.666874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db094b-d202-433a-85f2-7f93a7b8e93d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:13.760355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:13.761450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:13.763434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:13.763866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:13.765636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:13.766129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:13.768676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:13.797416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 129/262] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:13.806183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:13.827811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9d930-bd85-4ca5-86c8-728c776fa387 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:13.910371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:13.911253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:13.912199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:13.912577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.002924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:14.004515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:14.007270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:14.078528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8075a5-74ef-4fee-b918-4ec1bf2cbf2c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:14.131774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 134/263] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:14.138939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:14.141368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:14.159221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:14.160391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:14.164441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:14.164951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] PATH_INFO: `/v3/users/e72143a3716d4ef3bb8ad6e54353c383` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.166805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] RBAC: Authorizing `identity:get_user(user_id=e72143a3716d4ef3bb8ad6e54353c383)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:14.168495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:14.181989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 130/264] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:13 2022] GET /identity/v3/users/e72143a3716d4ef3bb8ad6e54353c383 => generated 283 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:14.197197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:14.369297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365158d-d734-4691-bafb-1d3695028e41 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:14.372628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:14.433735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f75f50-e033-4830-b9ad-a7396f3d13d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:14.458205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:14.460365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:14.461501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:14.462115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.469011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:14.470801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:14.473611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 135/265] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:14 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:14.480929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:14.518170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faedd1-c364-4198-996e-3abd2330330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:14.519131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faedd1-c364-4198-996e-3abd2330330a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:14.519558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faedd1-c364-4198-996e-3abd2330330a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:14.519978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faedd1-c364-4198-996e-3abd2330330a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.644556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dec6c-7393-42e8-8658-f286a3bd7c91 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:14.678859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:14.691464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faedd1-c364-4198-996e-3abd2330330a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:14.693525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faedd1-c364-4198-996e-3abd2330330a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:14.744375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:14.745060 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:14.745340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:14.745613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.746459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=e72143a3716d4ef3bb8ad6e54353c383) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:14.746774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:14.748277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:14.765699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 136/266] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:14 2022] GET /identity/v3/role_assignments?user.id=e72143a3716d4ef3bb8ad6e54353c383&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:14.897611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 131/267] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:30:14.903609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:14.916490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85198440-63c1-4358-bf17-068974f473d6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:14.920355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85198440-63c1-4358-bf17-068974f473d6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:14.921326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85198440-63c1-4358-bf17-068974f473d6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.922677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 132/268] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:14.928971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cece0-5963-4e34-a111-9547ffbd59ac None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:14.929314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cece0-5963-4e34-a111-9547ffbd59ac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:14.929541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cece0-5963-4e34-a111-9547ffbd59ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:14.968118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-198cece0-5963-4e34-a111-9547ffbd59ac None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:14.971503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-198cece0-5963-4e34-a111-9547ffbd59ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:14.979627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:15.098642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 133/269] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:15.100706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b9771-f457-4fb4-879a-0feb31f5ba8b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.101448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b9771-f457-4fb4-879a-0feb31f5ba8b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.101987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b9771-f457-4fb4-879a-0feb31f5ba8b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.104394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 134/270] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:15.110568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.111120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.111599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.126466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4de2-5c80-46fe-8571-a2fbc11aca9b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:15.137978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:15.139634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:15.167593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:15.202733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:15.204086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.204606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.204872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.206224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:15.207876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:15.210577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 137/271] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:14 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:15.211632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:15.264506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 135/272] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:15.266412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:15.366917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c568f-94f6-4187-8392-6561ce7ea5ac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:15.369195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:15.434705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:15.436465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.437090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.437629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.450483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3efd4-a5fb-4838-9820-f47007d18d72 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:15.454939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:15.464840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:15.465997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:15.466998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:15.469674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 138/273] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:15.471004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:15.508332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:15.509792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.510088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.510372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.511526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] RBAC: Adding query filter params (role.id=9508980101c44da48df022abc45e1e60, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=e72143a3716d4ef3bb8ad6e54353c383) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:15.511845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:15.514068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:15.523324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 136/274] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:15 2022] GET /identity/v3/role_assignments?user.id=e72143a3716d4ef3bb8ad6e54353c383&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=9508980101c44da48df022abc45e1e60 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:15.541944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:15.621151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:15.733868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b48ce-68c5-4531-af46-7f0a1c2637ba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:15.810868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:15.812168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.812670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.813823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.815521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:15.817267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:15.820294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 139/275] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:15.827583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:15.857587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c31a-f8d4-4e99-9d2c-9b4e50773834 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:15.880681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:15.920101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:15.921143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:15.921426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:15.921714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:15.922514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:15.922832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:15.924344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:15.940297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 137/276] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:15.950367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.020213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2d04-00b0-465f-89ad-120b074dcaad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.076697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cfa97-1d1c-4f50-9444-3636b97eb432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:16.077813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfa97-1d1c-4f50-9444-3636b97eb432 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.078233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfa97-1d1c-4f50-9444-3636b97eb432 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.079400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfa97-1d1c-4f50-9444-3636b97eb432 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.080151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cfa97-1d1c-4f50-9444-3636b97eb432 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:16.081737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cfa97-1d1c-4f50-9444-3636b97eb432 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:16.082401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:16.106562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af940b5f-6532-4448-8541-7aa38fcedabf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.108642 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 140/277] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:15 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:16.110529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599b342-01c0-4674-9b72-3884c129b0cc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.111138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599b342-01c0-4674-9b72-3884c129b0cc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.111666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599b342-01c0-4674-9b72-3884c129b0cc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.113574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 141/278] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:16.123987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5c73-53e4-40b6-a36e-6460766f2283 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.124341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5c73-53e4-40b6-a36e-6460766f2283 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.124651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5c73-53e4-40b6-a36e-6460766f2283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.154481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-131f5c73-53e4-40b6-a36e-6460766f2283 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:16.155884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-131f5c73-53e4-40b6-a36e-6460766f2283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:16.182673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c444b-9da5-41d2-833e-48f1bed53c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:16.184060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c444b-9da5-41d2-833e-48f1bed53c02 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.184565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c444b-9da5-41d2-833e-48f1bed53c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.185023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c444b-9da5-41d2-833e-48f1bed53c02 None admin] PATH_INFO: `/v3/users/f36c7d00a630492b8fc079e267e20899` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.204045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c444b-9da5-41d2-833e-48f1bed53c02 None admin] RBAC: Authorizing `identity:get_user(user_id=f36c7d00a630492b8fc079e267e20899)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:16.206101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c444b-9da5-41d2-833e-48f1bed53c02 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:16.208575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 138/279] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:15 2022] GET /identity/v3/users/f36c7d00a630492b8fc079e267e20899 => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:16.210904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120be1d3-36ad-4dc9-b176-63fef56220a2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.211396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120be1d3-36ad-4dc9-b176-63fef56220a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.211846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120be1d3-36ad-4dc9-b176-63fef56220a2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.214139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 139/280] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:16.215195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120be1d3-36ad-4dc9-b176-63fef56220a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.251109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 142/281] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:16.252616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.252914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.253200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.280836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:16.282639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:16.299231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:16.358196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120be1d3-36ad-4dc9-b176-63fef56220a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.386052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 143/282] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:16.397763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.399798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:16.400594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.401365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.401365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.402143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, user.id=f36c7d00a630492b8fc079e267e20899, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:16.402501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:16.404076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:16.410874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 140/283] 10.222.0.16 () {56 vars in 1308 bytes} [Thu May 26 13:30:16 2022] GET /identity/v3/role_assignments?user.id=f36c7d00a630492b8fc079e267e20899&scope.system=all&role.id=ee34d08904734afdb4939fd27d04b760 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:16.412495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.425437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:16.568491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e15b16-0bb9-4d14-a34d-21aeff2aea89 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.605206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0ad3-fed1-467b-b2b4-39ba028ca9c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.620942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:16.622065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.622352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.622630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.639800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:16.649501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:16.650436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:16.651209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 144/284] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:16.657274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.657563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:16.659086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.659607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.660011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.668585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:16.672240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:16.683768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:16.686343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:16.688262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 141/285] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:16.696249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.777122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3146ce-24ab-4782-9f0b-78d407492179 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.806777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:16.815378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:16.816072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:16.816419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:16.816770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:16.817554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:16.818426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:16.819946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:16.829927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 145/286] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:16.839942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:16.887403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e739ffc-136b-4162-ad49-9abd7be47fca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:16.942575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:16.988058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbe83a-0714-4098-8bc6-53ed8cf20f1b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:17.021208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:17.021967 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:17.022250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:17.022524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:17.023276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:17.023583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:17.025032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:17.031676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:17.033584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:17.034434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:17.034947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] PATH_INFO: `/v3/users/99bd9c42a52a4584a42b89bbf316e394` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:17.037347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 142/287] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:17.037837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] RBAC: Authorizing `identity:get_user(user_id=99bd9c42a52a4584a42b89bbf316e394)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:17.039691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:17.045394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:17.045948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 146/288] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:16 2022] GET /identity/v3/users/99bd9c42a52a4584a42b89bbf316e394 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:17.049701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:17.178583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ddaca-146e-4523-ad09-7200747a5806 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:17.224667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:17.235995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:17.237581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:17.237872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:17.238153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] PATH_INFO: `/v3/users/5360b748742a49cc816f9330b9c5a340` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:17.239512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] RBAC: Authorizing `identity:get_user(user_id=5360b748742a49cc816f9330b9c5a340)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:17.241549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:17.243843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 143/289] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:17 2022] GET /identity/v3/users/5360b748742a49cc816f9330b9c5a340 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:17.250675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:17.302380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9bb40-99f0-4544-9036-dd51b6aa7a79 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:17.396592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:17.399080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:17.399611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:17.400093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:17.402602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:17.407416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:17.410598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 147/290] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:17 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:17.422813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207a39a-2d2e-4e2e-887b-637f0849e07a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:17.424137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:17.441690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:17.511879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:17.515111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:17.515742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:17.516226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:17.610448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6399411-aba5-490d-9657-7e8bf92f6043 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:17.655657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:17.657756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:17.659733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:17.663633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9/users/99bd9c42a52a4584a42b89bbf316e394/roles/c71df34c0f1c4a2a8f0bff66914681f7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:17.672426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44454738f6541b6aa730195de6989a9, user_id=99bd9c42a52a4584a42b89bbf316e394, role_id=c71df34c0f1c4a2a8f0bff66914681f7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:17.674354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:17.696612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 148/291] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:30:17 2022] PUT /identity/v3/projects/d44454738f6541b6aa730195de6989a9/users/99bd9c42a52a4584a42b89bbf316e394/roles/c71df34c0f1c4a2a8f0bff66914681f7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:17.704916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:17.707002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:17.712701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:17.884180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 144/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:30:17.889409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:17.991492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:18.042380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd271b9f-cc31-4a7e-9980-84189046b689 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:18.092463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725fe6e-3ac4-4ac8-91a9-fe7e1c66ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:18.093328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fe6e-3ac4-4ac8-91a9-fe7e1c66ac63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:18.093609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fe6e-3ac4-4ac8-91a9-fe7e1c66ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:18.093887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fe6e-3ac4-4ac8-91a9-fe7e1c66ac63 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:18.095077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725fe6e-3ac4-4ac8-91a9-fe7e1c66ac63 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:18.096603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725fe6e-3ac4-4ac8-91a9-fe7e1c66ac63 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:18.098656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 149/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:17 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:18.105032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:18.255233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:18.265043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1f23-ad09-4ec2-9fed-54b5d111c595 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:18.322025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:18.322708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:18.323051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:18.323351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:18.324179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=5360b748742a49cc816f9330b9c5a340) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:18.324517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:18.325984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c9c31-b0fd-4071-8570-5679e5ea4170 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:18.334683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 145/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:18 2022] GET /identity/v3/role_assignments?user.id=5360b748742a49cc816f9330b9c5a340&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=ee34d08904734afdb4939fd27d04b760 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:18.951016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1513-8d08-4b8c-b4dd-6477e07ed50d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:18.951954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1513-8d08-4b8c-b4dd-6477e07ed50d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:18.952604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1513-8d08-4b8c-b4dd-6477e07ed50d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:18.954171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 150/295] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:18.960826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:18.961119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:18.961398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:18.984989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:18.986496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:19.034063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce8d9-dd57-41f0-8357-df4cb4fd4703 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.034734 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce8d9-dd57-41f0-8357-df4cb4fd4703 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.035212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ce8d9-dd57-41f0-8357-df4cb4fd4703 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.038696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 151/296] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:19.043409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.044199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.044817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.062979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:19.064516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:19.077572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 146/297] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:19.106409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.157821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 152/298] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:19.181006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.210257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:19.221536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799d9aa-5fc7-4549-9e76-a085dc225526 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.256566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:19.259886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.260803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.261160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.261509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.273381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:19.281343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:19.282700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:19.284129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 147/299] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:19.290476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.299796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b6bc0-4e97-44f1-b61a-4d5687562cdf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.340121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.341068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.341350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.341622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.353267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:19.361533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:19.363671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:19.366234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:19.370867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 153/300] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:19.372238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.397451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e2c6-f680-4541-9170-da2274558a8f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.439846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.440825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.441547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.441833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.443052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:19.443659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:19.445332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36219847-6853-4c4e-b0f3-9ef8f6326b17 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:19.453699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 148/301] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:19.455042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e296f-731e-4a3c-a849-0ac710578bb9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.455331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e296f-731e-4a3c-a849-0ac710578bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.455609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e296f-731e-4a3c-a849-0ac710578bb9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.457050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 149/302] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:19.461638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e296f-731e-4a3c-a849-0ac710578bb9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.481275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:19.497729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3e3a8-6e51-4d5a-8383-f9912ae73055 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.524026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:19.540831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.541581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.541822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.542121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.542867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:19.543172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:19.545014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e57c33-3c34-4eae-a59f-4cb59a92c8ed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:19.554743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 154/303] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:19.555996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.556287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.556597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.574639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:19.575826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:19.576379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e296f-731e-4a3c-a849-0ac710578bb9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.611464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.612355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.612640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.612914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] PATH_INFO: `/v3/users/a09bcaaa0a124b38a55a13f0a44f6efd` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.627222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] RBAC: Authorizing `identity:get_user(user_id=a09bcaaa0a124b38a55a13f0a44f6efd)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:19.628784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:19.631122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 150/304] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/users/a09bcaaa0a124b38a55a13f0a44f6efd => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:19.631607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.668327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 155/305] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:19.668327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.731624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:19.779744 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cab0b3-8467-4cd0-83ea-3252c2d3b258 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.799012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1db6-3a64-411e-9842-15c99010e38b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:19.814637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:19.823897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.824820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.825110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.825378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] PATH_INFO: `/v3/users/f36c7d00a630492b8fc079e267e20899` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.826735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] RBAC: Authorizing `identity:get_user(user_id=f36c7d00a630492b8fc079e267e20899)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:19.828315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:19.837185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 151/306] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/users/f36c7d00a630492b8fc079e267e20899 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:19.837185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.846602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:19.847352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:19.847701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:19.847983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:19.849266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] RBAC: Adding query filter params (role.id=c71df34c0f1c4a2a8f0bff66914681f7, user.id=a09bcaaa0a124b38a55a13f0a44f6efd, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:19.849742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:19.851222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:19.858887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 156/307] 10.222.0.16 () {56 vars in 1308 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/role_assignments?user.id=a09bcaaa0a124b38a55a13f0a44f6efd&scope.system=all&role.id=c71df34c0f1c4a2a8f0bff66914681f7 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:19.860195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:19.986958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd74f41-e4ab-4596-8d4c-443d88feac05 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:20.013509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d2c13-0aa0-49af-9e13-b72cf56b8c78 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:20.026804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:20.028094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.028094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.028557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.029913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:20.031497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:20.033542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 152/308] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:20.039894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:20.049410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:20.060546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70654587-9304-428d-b4d7-cd049cde6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:20.061723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70654587-9304-428d-b4d7-cd049cde6f59 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.062011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70654587-9304-428d-b4d7-cd049cde6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.062296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70654587-9304-428d-b4d7-cd049cde6f59 None admin] PATH_INFO: `/v3/system/users/f36c7d00a630492b8fc079e267e20899/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.063831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70654587-9304-428d-b4d7-cd049cde6f59 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f36c7d00a630492b8fc079e267e20899, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:20.065296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70654587-9304-428d-b4d7-cd049cde6f59 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:20.079853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 157/309] 10.222.0.16 () {58 vars in 1334 bytes} [Thu May 26 13:30:19 2022] PUT /identity/v3/system/users/f36c7d00a630492b8fc079e267e20899/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:20.175499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a7db-684c-4733-9ed9-b7fd4c5f87fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:20.220743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360c20b-a7bc-4693-85b8-ef63855d8649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:20.221434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360c20b-a7bc-4693-85b8-ef63855d8649 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.221718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360c20b-a7bc-4693-85b8-ef63855d8649 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.221998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360c20b-a7bc-4693-85b8-ef63855d8649 None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.222704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360c20b-a7bc-4693-85b8-ef63855d8649 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:20.224281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360c20b-a7bc-4693-85b8-ef63855d8649 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:20.243132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 153/310] 10.222.0.16 () {60 vars in 1134 bytes} [Thu May 26 13:30:20 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:20.283155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afc57-f46e-410b-b4f1-0de630d361f0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.283440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afc57-f46e-410b-b4f1-0de630d361f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.283732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afc57-f46e-410b-b4f1-0de630d361f0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.285684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 158/311] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:20.291752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.292092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.292334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.315044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:20.316364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:20.401503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:20.415104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 154/312] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:20.436916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89afc57-f46e-410b-b4f1-0de630d361f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:20.489845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:20.570838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89afc57-f46e-410b-b4f1-0de630d361f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:20.608081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:20.608938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.609222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.609505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.622142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:20.630831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:20.631813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:20.632641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 159/313] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:20.638584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:20.764903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165023ee-72f3-4a6b-8911-aa590f1a0c68 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:20.802837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:20.803512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:20.803791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:20.804063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:20.804836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:20.805139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:20.806522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:20.815724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 155/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:20.821468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:20.959635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efff6fc-8cb2-425f-ab91-22bf878dda99 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:20.967477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:20.999977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:21.001286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:21.001776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:21.002254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] PATH_INFO: `/v3/users/99bd9c42a52a4584a42b89bbf316e394` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:21.003726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] RBAC: Authorizing `identity:get_user(user_id=99bd9c42a52a4584a42b89bbf316e394)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:21.005566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:21.007631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 160/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:20 2022] GET /identity/v3/users/99bd9c42a52a4584a42b89bbf316e394 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:21.016520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:21.144256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:21.203827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea9eb7-19cc-4a59-80dc-41417c40a21a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:21.241715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:21.242373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:21.242655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:21.242933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:21.324329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:21.324329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:21.384739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:21.431630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 156/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:21.438208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:21.585726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291b98f-6090-4e74-a499-f200f1289fb5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:21.626443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5bdb0-10f6-40fe-836b-01fe0484cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:21.627314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5bdb0-10f6-40fe-836b-01fe0484cac3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:21.627592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5bdb0-10f6-40fe-836b-01fe0484cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:21.627862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5bdb0-10f6-40fe-836b-01fe0484cac3 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:21.629079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5bdb0-10f6-40fe-836b-01fe0484cac3 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:21.630582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5bdb0-10f6-40fe-836b-01fe0484cac3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:21.636413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 161/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:21 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:21.638516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:21.754407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e0ec-e3a2-4dff-93be-23e4019a5c13 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:21.793514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:21.794235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:21.794520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:21.794796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:21.795633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] RBAC: Adding query filter params (role.id=c71df34c0f1c4a2a8f0bff66914681f7, scope.project.id=d44454738f6541b6aa730195de6989a9, user.id=99bd9c42a52a4584a42b89bbf316e394) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:21.795941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:21.797447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aaa1c-bd18-4a73-b3d2-2f0e6898045b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:21.805042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 157/318] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:21 2022] GET /identity/v3/role_assignments?user.id=99bd9c42a52a4584a42b89bbf316e394&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=c71df34c0f1c4a2a8f0bff66914681f7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:22.372943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0e73d-610e-426a-a6b7-2efe253487f5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.373136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0e73d-610e-426a-a6b7-2efe253487f5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.373430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0e73d-610e-426a-a6b7-2efe253487f5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.374799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 162/319] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:22.380644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.381107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.381284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.398565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:22.399756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:22.416991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:22.471108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 158/320] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:22.493939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0e73d-610e-426a-a6b7-2efe253487f5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:22.542481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:22.616973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0e73d-610e-426a-a6b7-2efe253487f5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:22.658711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb7ea2-fb3c-42e0-9709-fe399ff379d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:22.659440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb7ea2-fb3c-42e0-9709-fe399ff379d3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.659724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb7ea2-fb3c-42e0-9709-fe399ff379d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.660002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb7ea2-fb3c-42e0-9709-fe399ff379d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.670728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb7ea2-fb3c-42e0-9709-fe399ff379d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eadb7ea2-fb3c-42e0-9709-fe399ff379d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:22.679297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:22.680400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:22.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 163/321] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:22.687816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:22.697106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfe64-2b74-4fb9-ae0f-348e3187aa2f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.697393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfe64-2b74-4fb9-ae0f-348e3187aa2f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.697680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfe64-2b74-4fb9-ae0f-348e3187aa2f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.699056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 164/322] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:22.704189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3030c57-6ebc-40fa-b701-b20b1a61906a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.704516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3030c57-6ebc-40fa-b701-b20b1a61906a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.704802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3030c57-6ebc-40fa-b701-b20b1a61906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.722417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b3030c57-6ebc-40fa-b701-b20b1a61906a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:22.723944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b3030c57-6ebc-40fa-b701-b20b1a61906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:22.796012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7e32-1b6c-4f54-be54-21038b414750 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:22.802661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 165/323] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:22.812586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cac5d0-3b0b-4b9a-bf24-7fba4f56db9a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.812879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cac5d0-3b0b-4b9a-bf24-7fba4f56db9a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.813161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cac5d0-3b0b-4b9a-bf24-7fba4f56db9a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.814783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 166/324] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:22.820171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.820491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.820770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.833683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:22.834614 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:22.834900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:22.835184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:22.835925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:22.836231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:22.837646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:22.838815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:22.840004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:22.846259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 159/325] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:22.846259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:22.863735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:22.872672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:22.925576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 167/326] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:22.926581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:22.962133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87190fe7-1984-4979-ae13-54038b02208e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.000349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.001122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.001410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.002061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.012450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:23.020406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:23.021743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:23.022554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 160/327] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:23.023232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:23.039411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c8dae-1971-4dd9-9e6c-ffbcf32fc2c6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.089158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.090065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.090342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.090610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] PATH_INFO: `/v3/users/a09bcaaa0a124b38a55a13f0a44f6efd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.091833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] RBAC: Authorizing `identity:get_user(user_id=a09bcaaa0a124b38a55a13f0a44f6efd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.093396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.094975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 168/328] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:22 2022] GET /identity/v3/users/a09bcaaa0a124b38a55a13f0a44f6efd => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:23.097219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:23.107494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:23.130484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:23.137625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af869e-8dd1-4ed2-b563-9bb42904b579 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.172525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.173236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.173558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.173797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.175163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.178853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.182428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 161/329] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:23.183836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:23.216153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3c683-13d4-4e0d-a796-18d529fecffa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.261265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.261940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.262228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.262508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.263259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:23.263575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.265005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.274089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 169/330] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:23.276552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:23.303248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25779010-d94e-45ca-bc55-b89db87b21e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.340611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.341489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.341776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.342058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] PATH_INFO: `/v3/system/users/a09bcaaa0a124b38a55a13f0a44f6efd/roles/c71df34c0f1c4a2a8f0bff66914681f7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.343604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a09bcaaa0a124b38a55a13f0a44f6efd, role_id=c71df34c0f1c4a2a8f0bff66914681f7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.345078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.355797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 162/331] 10.222.0.16 () {58 vars in 1334 bytes} [Thu May 26 13:30:23 2022] PUT /identity/v3/system/users/a09bcaaa0a124b38a55a13f0a44f6efd/roles/c71df34c0f1c4a2a8f0bff66914681f7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:23.356635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:23.366699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:23.399600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:23.417729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc966406-e5ad-49c0-90a7-e15318cef088 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.460428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.461131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.461418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.461703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.462421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.464129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.490204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 170/332] 10.222.0.16 () {60 vars in 1134 bytes} [Thu May 26 13:30:23 2022] POST /identity/v3/groups => generated 226 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:23.490400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1897dc-da03-4b6b-b6ed-d8f0eac33606 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.538001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c316c9-9279-449a-b4af-98c15e13524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.539053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c316c9-9279-449a-b4af-98c15e13524f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.539372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c316c9-9279-449a-b4af-98c15e13524f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.539644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c316c9-9279-449a-b4af-98c15e13524f None admin] PATH_INFO: `/v3/users/f36c7d00a630492b8fc079e267e20899` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.540985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c316c9-9279-449a-b4af-98c15e13524f None admin] RBAC: Authorizing `identity:get_user(user_id=f36c7d00a630492b8fc079e267e20899)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.542512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c316c9-9279-449a-b4af-98c15e13524f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.544499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 163/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:23 2022] GET /identity/v3/users/f36c7d00a630492b8fc079e267e20899 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:23.550600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:23.680730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d547b-8465-4312-8b64-61a5d0f52836 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:23.723011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:23.724087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:23.724410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:23.724690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:23.725533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] RBAC: Adding query filter params (role.id=ee34d08904734afdb4939fd27d04b760, user.id=f36c7d00a630492b8fc079e267e20899, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:23.725844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:23.728480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cdc1b-3163-4be8-935c-a3fa2b10a47b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:23.744254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 171/334] 10.222.0.16 () {56 vars in 1308 bytes} [Thu May 26 13:30:23 2022] GET /identity/v3/role_assignments?user.id=f36c7d00a630492b8fc079e267e20899&scope.system=all&role.id=ee34d08904734afdb4939fd27d04b760 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:25.788605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4205a-31ef-4928-9d4f-4ce03237c51d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:25.789173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4205a-31ef-4928-9d4f-4ce03237c51d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:25.789173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4205a-31ef-4928-9d4f-4ce03237c51d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:25.790898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 164/335] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:25.797156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:25.797681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:25.797763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:25.817115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:25.827488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:25.833710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:25.934244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 172/336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:25.974270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4205a-31ef-4928-9d4f-4ce03237c51d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:26.125399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:26.170490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4205a-31ef-4928-9d4f-4ce03237c51d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:26.237292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49839ab-a145-4ed8-a862-41ec7127c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:26.238541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49839ab-a145-4ed8-a862-41ec7127c322 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:26.239547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49839ab-a145-4ed8-a862-41ec7127c322 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:26.240045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49839ab-a145-4ed8-a862-41ec7127c322 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:26.269252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49839ab-a145-4ed8-a862-41ec7127c322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a49839ab-a145-4ed8-a862-41ec7127c322 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:26.278129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:26.279088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:26.280141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 165/337] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:26.288707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:26.352352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be49da-42db-4993-84fb-bed419b7eb59 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:26.352352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be49da-42db-4993-84fb-bed419b7eb59 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:26.352352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be49da-42db-4993-84fb-bed419b7eb59 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:26.352682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 166/338] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:26.362167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:26.362488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:26.362729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:26.409576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:26.410760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:26.536423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c926-56fd-43e9-b4a4-9d7c880fde09 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:26.620454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 167/339] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:26.637308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:26.639484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:26.640092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:26.641158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:26.642222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:26.642750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:26.646025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:26.660518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:26.667789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 173/340] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:26.672490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:26.673781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:26.800679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:26.874454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a51ba-fc08-4841-a9f7-df04885e0db4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:26.937027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f810b-6a1e-43fc-a806-6284c45842b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:26.938113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f810b-6a1e-43fc-a806-6284c45842b6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:26.938528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f810b-6a1e-43fc-a806-6284c45842b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:26.938943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f810b-6a1e-43fc-a806-6284c45842b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:26.975544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f810b-6a1e-43fc-a806-6284c45842b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:26.975779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1ee7d-7c90-4a38-a3e9-2143aa2bce7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-865f810b-6a1e-43fc-a806-6284c45842b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:27.012338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:27.013342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:27.014159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:27.021262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 168/341] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:27.022999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048b3f3-8197-4e86-bfbd-b12966d0af20 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:27.024430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048b3f3-8197-4e86-bfbd-b12966d0af20 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:27.028456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048b3f3-8197-4e86-bfbd-b12966d0af20 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:27.036367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 169/342] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:27.037732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048b3f3-8197-4e86-bfbd-b12966d0af20 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:27.134601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a0a8b-ec5c-4883-a68f-52029c4cabd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:27.136809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a0a8b-ec5c-4883-a68f-52029c4cabd6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:27.137407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a0a8b-ec5c-4883-a68f-52029c4cabd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:27.137899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a0a8b-ec5c-4883-a68f-52029c4cabd6 None admin] PATH_INFO: `/v3/users/a09bcaaa0a124b38a55a13f0a44f6efd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:27.149427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a0a8b-ec5c-4883-a68f-52029c4cabd6 None admin] RBAC: Authorizing `identity:get_user(user_id=a09bcaaa0a124b38a55a13f0a44f6efd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:27.151322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a0a8b-ec5c-4883-a68f-52029c4cabd6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:27.153761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 174/343] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:30:26 2022] GET /identity/v3/users/a09bcaaa0a124b38a55a13f0a44f6efd => generated 293 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:27.155436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:27.155978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:27.156557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:27.222226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:27.225096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:27.436339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:27.458349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 175/344] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:27.459812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:27.499170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048b3f3-8197-4e86-bfbd-b12966d0af20 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:27.586292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:27.674499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:27.676345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:27.677040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:27.677659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:27.680676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:27.681673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:27.687606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:27.720127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 170/345] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 683 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:27.724803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:27.881772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19034b0c-494e-4323-b626-d05d5d548085 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.012598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.014320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.014607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.014889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.015744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] RBAC: Adding query filter params (role.id=c71df34c0f1c4a2a8f0bff66914681f7, user.id=a09bcaaa0a124b38a55a13f0a44f6efd, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:28.016062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:28.017651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0d6b5-e3b4-4ad4-bfc7-341220175d93 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:28.034935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 176/346] 10.222.0.16 () {56 vars in 1308 bytes} [Thu May 26 13:30:27 2022] GET /identity/v3/role_assignments?user.id=a09bcaaa0a124b38a55a13f0a44f6efd&scope.system=all&role.id=c71df34c0f1c4a2a8f0bff66914681f7 => generated 510 bytes in 576 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:28.037123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6441748-a921-441f-baf6-c1ba7a4dbdcd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.037611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6441748-a921-441f-baf6-c1ba7a4dbdcd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.044349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6441748-a921-441f-baf6-c1ba7a4dbdcd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.046505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 177/347] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:28.047604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6441748-a921-441f-baf6-c1ba7a4dbdcd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:28.061694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2691c0-1285-454b-a9b8-9538669e0e83 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.117117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02ef69-08be-4b44-a16b-b535caa09f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.117964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02ef69-08be-4b44-a16b-b535caa09f19 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.118247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02ef69-08be-4b44-a16b-b535caa09f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.118526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02ef69-08be-4b44-a16b-b535caa09f19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.151100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02ef69-08be-4b44-a16b-b535caa09f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fb02ef69-08be-4b44-a16b-b535caa09f19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:28.159084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:28.160080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:28.161703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:28.171359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 171/348] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:28.172054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:28.172972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.173204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.173490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.221511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:28.223103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:28.240489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6441748-a921-441f-baf6-c1ba7a4dbdcd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.285523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345967e-1f13-4257-bf12-f0879e74f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.286980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345967e-1f13-4257-bf12-f0879e74f003 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.288488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345967e-1f13-4257-bf12-f0879e74f003 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.288910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345967e-1f13-4257-bf12-f0879e74f003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.299727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:28.358958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 172/349] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:28.360390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:28.410583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345967e-1f13-4257-bf12-f0879e74f003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:28.413602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345967e-1f13-4257-bf12-f0879e74f003 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:28.509857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:28.527435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afee150-520a-4c7b-a060-59d9fe169fa2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.581483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11933652-28e7-4383-8470-06b888938921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.582586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11933652-28e7-4383-8470-06b888938921 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.582586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11933652-28e7-4383-8470-06b888938921 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.583474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11933652-28e7-4383-8470-06b888938921 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.584748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11933652-28e7-4383-8470-06b888938921 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:28.585048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11933652-28e7-4383-8470-06b888938921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:28.586475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11933652-28e7-4383-8470-06b888938921 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:28.598562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 178/350] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:28.599816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0e848-fd02-4f31-9e2f-f2ce3df2d926 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.600115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0e848-fd02-4f31-9e2f-f2ce3df2d926 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.600580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0e848-fd02-4f31-9e2f-f2ce3df2d926 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.601908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 179/351] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:28.603205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 173/352] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:28.604492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11933652-28e7-4383-8470-06b888938921 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:28.605129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0e848-fd02-4f31-9e2f-f2ce3df2d926 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:28.675571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:28.725515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0e848-fd02-4f31-9e2f-f2ce3df2d926 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.740177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11933652-28e7-4383-8470-06b888938921 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.766509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3427ac-a93e-4283-81e3-e455c5a19880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.767718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3427ac-a93e-4283-81e3-e455c5a19880 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.768211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3427ac-a93e-4283-81e3-e455c5a19880 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.769256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3427ac-a93e-4283-81e3-e455c5a19880 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.785034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3427ac-a93e-4283-81e3-e455c5a19880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9e3427ac-a93e-4283-81e3-e455c5a19880 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:28.794869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:28.795822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:28.796644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:28.797252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.797340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 180/353] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:28.797406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.797406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.797406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.797587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.797587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.797587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:28.798346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:28.800040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:28.804162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 174/354] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:28 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:28.804648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:28.818050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:28.819413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:28.830261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:28.910540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 181/355] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:28.912546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:28.948896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb662d95-e1ff-4720-bfef-38e5d9688b90 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:28.957940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:28.995380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:28.996471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:28.996967 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:28.997445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.066102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1687f-870f-49bd-9b7d-9058314835e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.090015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.091492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:29.116790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.117869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.118302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.118741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.120877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:29.120877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.123009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2d29e-7548-4327-a687-a35b094cbbd3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:29.136370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 182/356] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:29.138010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086d6a-2662-40de-8332-e3bb057fb7ce None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.138489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086d6a-2662-40de-8332-e3bb057fb7ce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.138974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086d6a-2662-40de-8332-e3bb057fb7ce None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.141472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 183/357] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:29.142430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086d6a-2662-40de-8332-e3bb057fb7ce None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.154244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:29.233713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 175/358] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:29.234978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.246125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:29.297121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086d6a-2662-40de-8332-e3bb057fb7ce None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.343399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.344259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.344575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.344854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.352464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.353927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:29.355679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 184/359] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:29.356490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.378685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca7f38-0cb5-4acf-b2e9-daa584c2ef82 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.427780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c97c14-5995-4b03-8917-13dd477e5a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.428638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c97c14-5995-4b03-8917-13dd477e5a73 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.428921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c97c14-5995-4b03-8917-13dd477e5a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.429299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c97c14-5995-4b03-8917-13dd477e5a73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.440522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c97c14-5995-4b03-8917-13dd477e5a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:29.441850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-85c97c14-5995-4b03-8917-13dd477e5a73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:29.449853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:29.450803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:29.451829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 176/360] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:29.457308 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.457489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.458507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.476410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:29.477755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:29.485496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e378f-194a-4b9f-8272-5047f4b86c4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.526495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.526725 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.526981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.527219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.562332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:29.576254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 177/361] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:29.577434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.627177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.630507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:29.722444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:29.735365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218c09c-febd-42ab-9bfb-5cd058ba1595 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.764528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 185/362] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:29.765693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.779890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.781449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.781449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.781449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.783498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.784151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:29.791061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 178/363] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:29.791061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.872096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:29.894624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f914fc-0198-448a-b1a9-2ca3992e5def None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.931255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8597b-bd32-4303-a236-a4efa717632a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:29.945326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.946288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.946571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.946848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.947681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:29.948035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.950075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:29.956159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 186/364] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=ee34d08904734afdb4939fd27d04b760 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:29.957178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:29.987399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:29.988224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:29.988609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:29.989493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:29.990383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:29.990828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:29.992509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.007070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 179/365] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.007070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.011029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.112088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287eef8-945b-4d3d-b228-261f58338517 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.152186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd3e48-c8b2-40ba-aefc-01e4ca7f8257 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.153678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.154559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.154848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.155129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.171038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:30.179299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:30.180226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:30.181070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:30.185440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 187/366] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:30.186080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.186361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.196601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.197575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.197865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.198207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.199547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.201109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.202983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 180/367] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.203767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.306711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.314357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90d4bd-357d-4dc3-a935-3adec3aa790e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.318597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e9d6-86f0-498b-b30e-29595012ae56 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.352476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.353336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.353616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.353872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.354874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.355816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.355938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.356277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.357280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.358761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.360540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 188/368] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.361356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.430197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.437853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.439350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.483827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0a5be-6099-483a-b763-a250abe61289 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.523467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.524136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.524452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.524758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.525505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:30.525833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.527221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.536815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 189/369] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.537853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.560351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 181/370] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:30.560351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.573609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.661230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3a42b-3078-44fc-89b4-3a647d1a1cfe None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.669557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.682621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf0328-85a7-4539-9051-c499d7f91831 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.695691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.696877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.697156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.697430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.698629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.700068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.701745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 190/371] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.702511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.729577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.730232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.730232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.730806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.731612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=d44454738f6541b6aa730195de6989a9) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:30.731920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.733439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.748336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 182/372] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=ee34d08904734afdb4939fd27d04b760 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.748336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:30.850086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9367f95-2ed5-4744-87b6-334c0f4d1309 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.853139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:30.903194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d7ea1-1e0f-43e1-9ff8-7b9b962d7a69 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:30.911749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.912485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.912778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.914237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.975199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:30.976979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:30.977268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:30.977548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:30.980902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:30.982889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:30.985018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 183/373] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:30.986160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:31.022944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:31.025022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:31.027665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:31.135110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3314426-1dbb-48f7-ac59-721880bc1907 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:31.171260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:31.178832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:31.179673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:31.179959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:31.180611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:31.181737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=9508980101c44da48df022abc45e1e60, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:31.182055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:31.183528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:31.184421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 191/374] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:31.185309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:31.191512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 184/375] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:30 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=9508980101c44da48df022abc45e1e60 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:31.197281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:31.412927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:31.419298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267cf99-d5ff-457d-a0fb-e11c38be1362 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:31.433583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c2e3f-e87d-4c3e-8c3b-8829b3393dd3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:31.468404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:31.469378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:31.469664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:31.469941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:31.471213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:31.472816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:31.474910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 192/376] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:31 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:31.480735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:31.481708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:31.483108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:31.483542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:31.483975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] PATH_INFO: `/v3/projects/56278f94460f422387fbeb96048d2270` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:31.485994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] RBAC: Authorizing `identity:get_project(project_id=56278f94460f422387fbeb96048d2270)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:31.488430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:31.492187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 185/377] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:31 2022] GET /identity/v3/projects/56278f94460f422387fbeb96048d2270 => generated 344 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:31.497995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:31.548776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:31.588800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474debd6-bf1f-4582-871e-6df97dd3b567 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:31.625434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:31.626118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:31.626415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:31.626692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:31.627530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=9508980101c44da48df022abc45e1e60, scope.project.id=d44454738f6541b6aa730195de6989a9) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:31.627845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:31.629454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:31.636223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 193/378] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:31 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=9508980101c44da48df022abc45e1e60 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:31.747952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6a2a9-a9ec-4715-b66f-fcf812d47694 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:31.824256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4a701-fb92-4da6-9788-a2fd9e34134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:31.825180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4a701-fb92-4da6-9788-a2fd9e34134e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:31.825459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4a701-fb92-4da6-9788-a2fd9e34134e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:31.825734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4a701-fb92-4da6-9788-a2fd9e34134e None admin] PATH_INFO: `/v3/groups/8d60acee023f4252a333d2df06622c5b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:31.838025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4a701-fb92-4da6-9788-a2fd9e34134e None admin] RBAC: Authorizing `identity:get_group(group_id=8d60acee023f4252a333d2df06622c5b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:31.839555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4a701-fb92-4da6-9788-a2fd9e34134e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:31.841714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 186/379] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:31 2022] GET /identity/v3/groups/8d60acee023f4252a333d2df06622c5b => generated 229 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:31.850188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:31.985157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f058-4671-416a-8578-b6a01aed6a40 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:32.013029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:32.030078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:32.032888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:32.032888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:32.032888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:32.032888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] RBAC: Adding query filter params (group.id=8d60acee023f4252a333d2df06622c5b, role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=56278f94460f422387fbeb96048d2270) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:32.032888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:32.033953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219dc49-63c5-4514-b496-273202081f18 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:32.042317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 194/380] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:31 2022] GET /identity/v3/role_assignments?group.id=8d60acee023f4252a333d2df06622c5b&scope.project.id=56278f94460f422387fbeb96048d2270&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:32.500967 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291e7f8-c5e1-4f27-9501-6aca03e54bce None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:32.501492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291e7f8-c5e1-4f27-9501-6aca03e54bce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:32.501582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291e7f8-c5e1-4f27-9501-6aca03e54bce None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:32.502954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 187/381] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:32.508897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:32.509331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:32.509518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:32.532199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:32.534448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:32.666425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 195/382] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:32.705430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291e7f8-c5e1-4f27-9501-6aca03e54bce None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:32.759320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:32.889061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291e7f8-c5e1-4f27-9501-6aca03e54bce None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:32.941541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:32.942705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:32.943208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:32.943692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:32.963293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:32.971024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:32.971920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:32.972756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 188/383] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:32.981415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:33.130365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb0445-f56f-4252-988a-c270162378f9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:33.148785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:33.177345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:33.178249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:33.178539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:33.178820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:33.179607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:33.179921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:33.181413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:33.195854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 196/384] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:33.201541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:33.370020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:33.416808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a3952-9672-4c5b-b17e-12fb758dd766 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:33.481743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1a3d9-7893-4d79-a1dc-b50191496aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:33.482602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1a3d9-7893-4d79-a1dc-b50191496aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:33.482889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1a3d9-7893-4d79-a1dc-b50191496aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:33.483250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1a3d9-7893-4d79-a1dc-b50191496aa0 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:33.489154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1a3d9-7893-4d79-a1dc-b50191496aa0 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:33.490662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1a3d9-7893-4d79-a1dc-b50191496aa0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:33.492519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 189/385] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:33 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:33.498033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:33.614255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef922c7-9c47-4c80-a1ca-9790e9fe224a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:33.614804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef922c7-9c47-4c80-a1ca-9790e9fe224a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:33.615110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef922c7-9c47-4c80-a1ca-9790e9fe224a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:33.616484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 190/386] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:33.620514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced309c8-41bf-4e37-b852-bb14117eae25 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:33.630713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:33.631050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:33.631453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:33.665203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:33.666575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:33.695851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:33.696801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:33.697129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:33.697418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:33.699842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:33.701324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:33.711519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 197/387] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:33 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:33.713185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:33.821354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 191/388] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:33.848699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:33.859483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:33.900574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d5f43-420b-421b-9f04-05bdfaefb131 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.007889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e436-eebc-4ee5-9363-9d1a0e8205b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.008788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e436-eebc-4ee5-9363-9d1a0e8205b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.009083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e436-eebc-4ee5-9363-9d1a0e8205b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.009360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e436-eebc-4ee5-9363-9d1a0e8205b8 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.011264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:34.013772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e436-eebc-4ee5-9363-9d1a0e8205b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b58b1b03bb4947b4be69f8dc27be3cb4, group_id=39e50f0cb88b4dd5b6126200afbf5504, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:34.015245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e436-eebc-4ee5-9363-9d1a0e8205b8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:34.017496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986bba9-d28b-4754-a0e2-e77d3b13533b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.031811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 198/389] 10.222.0.16 () {58 vars in 1442 bytes} [Thu May 26 13:30:33 2022] PUT /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:34.033470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817c0d-e446-4734-bf55-28c31475e048 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.033858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817c0d-e446-4734-bf55-28c31475e048 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.034278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817c0d-e446-4734-bf55-28c31475e048 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.040283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 199/390] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:34.042045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.043575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.043880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.084991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:34.086485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:34.091974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.093336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.093624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.093899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.123788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:34.140927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:34.141899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:34.143601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 192/391] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:34.151251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:34.229896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 200/392] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:34.251691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:34.296814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:34.340789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037420e-ed80-4d8c-9eff-36742f82a775 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.375671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e9662-4a59-4d51-b9b3-81bd4473bec1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.384138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.385307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.385307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.388339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.388339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:34.388701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:34.389995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e89114-6534-488d-aa46-8241999f3c77 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:34.404347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 193/393] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:34.404347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43db5e-2d6e-49d2-bda4-b01dc413f33d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.404614 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43db5e-2d6e-49d2-bda4-b01dc413f33d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.404868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43db5e-2d6e-49d2-bda4-b01dc413f33d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.407356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 194/394] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:34.413790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43db5e-2d6e-49d2-bda4-b01dc413f33d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:34.416940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:34.432607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade52b0-663c-4b62-a2c6-1b7307c5b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.434354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade52b0-663c-4b62-a2c6-1b7307c5b6aa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.435067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade52b0-663c-4b62-a2c6-1b7307c5b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.435527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade52b0-663c-4b62-a2c6-1b7307c5b6aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.452070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade52b0-663c-4b62-a2c6-1b7307c5b6aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7ade52b0-663c-4b62-a2c6-1b7307c5b6aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:34.463565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:34.464592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:34.465394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:34.468288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 201/395] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:34.470660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.471149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.471614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.500555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:34.501810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:34.573255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43db5e-2d6e-49d2-bda4-b01dc413f33d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.661446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 202/396] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:34.662460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:34.663679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.664607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.664892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.665173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.668705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:34.671204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:34.686931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 195/397] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:34 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:34.689895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:34.751193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:34.830309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:34.890939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c1144-bb50-4a1c-88fa-b1fca8f799c6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.924908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283522c-49bc-4ea2-86ef-c903021d3306 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:34.990214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.993736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:34.995517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:34.996053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:34.997176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:34.998089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.001026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.001642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.002117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.005762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:35.006969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.007721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.011649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 196/398] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:34 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.012223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e668e0-f0b4-4d9d-a8e7-93ed726f8f5b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.014820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.023591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 203/399] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.025221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323646-04a9-4b60-b73d-a05e992a8bc3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.025511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323646-04a9-4b60-b73d-a05e992a8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.025792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323646-04a9-4b60-b73d-a05e992a8bc3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.028351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 204/400] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:35.028968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f323646-04a9-4b60-b73d-a05e992a8bc3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.137352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1da2e-d7e6-44ab-8760-7f2f7419f796 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.137678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:35.149617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f323646-04a9-4b60-b73d-a05e992a8bc3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.179244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.180066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.180401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.180687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.187379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cdc24-db9b-48a5-beef-72edb81682e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.188401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cdc24-db9b-48a5-beef-72edb81682e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.188702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cdc24-db9b-48a5-beef-72edb81682e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.188928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:35.189039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cdc24-db9b-48a5-beef-72edb81682e5 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.191340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cdc24-db9b-48a5-beef-72edb81682e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d44454738f6541b6aa730195de6989a9, group_id=39e50f0cb88b4dd5b6126200afbf5504, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.193133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:35.193264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cdc24-db9b-48a5-beef-72edb81682e5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:35.202234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:35.203273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:35.204074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 197/401] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:35.205408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.224576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 205/402] 10.222.0.16 () {58 vars in 1442 bytes} [Thu May 26 13:30:35 2022] PUT /identity/v3/projects/d44454738f6541b6aa730195de6989a9/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:35.224576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.224576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.224576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.243092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:35.244457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:35.323294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26b351-5b02-40e5-b0d5-6c89bf98135d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.345353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 206/403] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:35.346595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.377485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.380560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.381314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.385068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.387345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.389022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.390786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 198/404] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.391652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.496799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:35.496990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:35.510490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521a353-527f-4c1a-949c-a2e205276e08 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.529550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07c1a1-68f1-4da2-9136-150a854828ee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.551984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.553086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.553708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.554191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.555213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:35.555726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.557615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.565010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.565808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.566100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.566381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.568993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 207/405] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.570222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.577954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:35.586894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:35.587904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 199/406] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:35.593437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.694799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eced57f-6a68-4035-a3d3-e73af40bcc3e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.695696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf00e3b-2485-4446-8fa5-d8c52c1dd25b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.729359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:35.736957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.738188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.738736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.738928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.739353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.739594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.739879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.740156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.741322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.741489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.742922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.743313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.744877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 200/407] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.745629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 208/408] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.745846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.755115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.769254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:35.868747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca949d5-58f5-4b33-93c8-ad04d7f6390d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.871992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b84b19-e60a-4df0-b8ef-935225830ff5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:35.905619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.906296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.906636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.906921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.907774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:35.908093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.909396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:35.909642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.911265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:35.911756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:35.912178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:35.913623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:35.915283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:35.918079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 209/409] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.923924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 201/410] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:35.924538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:35.924818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:36.007061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:36.017654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:36.052464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08987c-d38a-4422-ae29-b5cae9f46a9b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:36.064146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d594970-e3d7-461c-b1eb-30441753b631 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:36.089640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:36.090829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.091260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.091701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/9508980101c44da48df022abc45e1e60` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.094216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] RBAC: Authorizing `identity:create_grant(project_id=b58b1b03bb4947b4be69f8dc27be3cb4, group_id=39e50f0cb88b4dd5b6126200afbf5504, role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:36.095871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:36.107925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:36.109455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.109749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.110201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/9508980101c44da48df022abc45e1e60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.112858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] RBAC: Authorizing `identity:create_grant(project_id=d44454738f6541b6aa730195de6989a9, group_id=39e50f0cb88b4dd5b6126200afbf5504, role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:36.115174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:36.115364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 210/411] 10.222.0.16 () {58 vars in 1442 bytes} [Thu May 26 13:30:35 2022] PUT /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/9508980101c44da48df022abc45e1e60 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:36.116384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:36.141214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 202/412] 10.222.0.16 () {58 vars in 1442 bytes} [Thu May 26 13:30:35 2022] PUT /identity/v3/projects/d44454738f6541b6aa730195de6989a9/groups/39e50f0cb88b4dd5b6126200afbf5504/roles/9508980101c44da48df022abc45e1e60 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:36.360034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d57762-06b4-4095-8406-b3d857571a6e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:36.403913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:36.404833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.405121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.405403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] PATH_INFO: `/v3/groups/8d60acee023f4252a333d2df06622c5b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.406591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] RBAC: Authorizing `identity:get_group(group_id=8d60acee023f4252a333d2df06622c5b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:36.408030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:36.409859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 211/413] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:36 2022] GET /identity/v3/groups/8d60acee023f4252a333d2df06622c5b => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:36.415375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:36.537564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912087ea-e7a2-4c1e-8f43-a09a195f91fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:36.556704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:36.574687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380695c-5be7-4d02-abb8-448d4ffa640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:36.575575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380695c-5be7-4d02-abb8-448d4ffa640e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.576205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380695c-5be7-4d02-abb8-448d4ffa640e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.576524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380695c-5be7-4d02-abb8-448d4ffa640e None admin] PATH_INFO: `/v3/projects/56278f94460f422387fbeb96048d2270` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.578194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380695c-5be7-4d02-abb8-448d4ffa640e None admin] RBAC: Authorizing `identity:get_project(project_id=56278f94460f422387fbeb96048d2270)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:36.579653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380695c-5be7-4d02-abb8-448d4ffa640e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:36.586517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 203/414] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:36 2022] GET /identity/v3/projects/56278f94460f422387fbeb96048d2270 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:36.587991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:36.602887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:36.732336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c1c55-fbd3-4b0a-92c9-84d9086bb334 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.732336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c1c55-fbd3-4b0a-92c9-84d9086bb334 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.732336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c1c55-fbd3-4b0a-92c9-84d9086bb334 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.732983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 204/415] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:36.744343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.744343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.744343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.753616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a225b7-9bbd-45b4-8dd8-d055869dfae0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:36.777403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:36.778686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:36.792220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:36.793155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:36.793441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:36.793723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] PATH_INFO: `/v3/projects/56278f94460f422387fbeb96048d2270/groups/8d60acee023f4252a333d2df06622c5b/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:36.796241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] RBAC: Authorizing `identity:create_grant(project_id=56278f94460f422387fbeb96048d2270, group_id=8d60acee023f4252a333d2df06622c5b, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:36.797903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:36.829650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 212/416] 10.222.0.16 () {58 vars in 1442 bytes} [Thu May 26 13:30:36 2022] PUT /identity/v3/projects/56278f94460f422387fbeb96048d2270/groups/8d60acee023f4252a333d2df06622c5b/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:30:36.946022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 205/417] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:36.986799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:37.123562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:37.197907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd86669-89b6-48ba-b6d0-33fb1484df36 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:37.255653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:37.256420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:37.256707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:37.256992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:37.278344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:37.287031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:37.288000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:37.289336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 213/418] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:37.295473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:37.497589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4569ce5-cdab-4fc1-a0a7-70a9de5496c8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:37.515289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:37.581074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:37.581783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:37.584338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:37.584338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:37.584338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:37.584581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:37.585873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ebcfb-f30e-4362-a1e4-028399d429bb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:37.595554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 206/419] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:37.605488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:37.693811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4011c-3d39-4c8d-8718-b3fcd36fd01e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:37.694429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4011c-3d39-4c8d-8718-b3fcd36fd01e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:37.694922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4011c-3d39-4c8d-8718-b3fcd36fd01e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:37.696775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 207/420] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:37.702282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:37.702530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:37.702815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:37.739933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:37.743138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:37.832548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e461140-dbdd-42a9-a144-85317db04591 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:37.876703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:37.900413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:37.901591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:37.902747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:37.903357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:37.931269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 208/421] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:37.979545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:38.005557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:38.007307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:38.171880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c57eb-1caf-47da-958d-87596c585a7f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:38.185403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:38.220101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 214/422] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:30:38.221134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:38.239576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:38.240823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:38.241331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:38.241806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:38.254621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:38.263597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:38.264584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 13:30:38.265471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:38.268404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 209/423] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:38.280463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:38.309946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:38.470413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f74b07-3f1b-4b0e-aefa-81ce346a5674 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:38.496203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b0809-6599-45e3-a46d-53a0b94788df None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:38.534315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:38.536504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:38.537005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:38.537936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:38.538860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:38.539371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:38.541046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:38.551318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 210/424] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:30:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:38.567807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:38.604501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:38.629678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:38.630903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:38.631200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:38.631487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:38.637915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:38.639404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:38.641365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 215/425] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:38 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:30:38.653266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:38.684021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16731c0a-287c-473b-bf9c-f16072891d98 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:38.732586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:38.736325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:38.736325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:38.736325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:38.821112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:38.832411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:38.856791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:38.934849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:38.960471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42e10f-5ff3-4525-a2a0-be9b7d6ed171 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:38.984215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 211/426] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:30:38.991657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.047132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daea75f-dcd6-4fe7-88a9-69868e61cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.048043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea75f-dcd6-4fe7-88a9-69868e61cd1f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.048364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea75f-dcd6-4fe7-88a9-69868e61cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.048656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea75f-dcd6-4fe7-88a9-69868e61cd1f None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.049952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea75f-dcd6-4fe7-88a9-69868e61cd1f None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:39.052921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea75f-dcd6-4fe7-88a9-69868e61cd1f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:39.055066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 216/427] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:38 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:39.057086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622909-9f07-4a6b-8113-fea72f750243 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.057384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622909-9f07-4a6b-8113-fea72f750243 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.057673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622909-9f07-4a6b-8113-fea72f750243 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.060407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 217/428] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:39.061901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba40f4-ce28-43fc-87f6-06dd6c96dd90 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.062197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba40f4-ce28-43fc-87f6-06dd6c96dd90 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.062483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba40f4-ce28-43fc-87f6-06dd6c96dd90 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.064223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 218/429] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:39.065025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba40f4-ce28-43fc-87f6-06dd6c96dd90 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.121805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9fbac-0b3f-4fcf-8131-75c004d5c784 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.155142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:39.164459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceb206-764b-4b82-b149-bc4d96950a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.165513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceb206-764b-4b82-b149-bc4d96950a83 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.165793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceb206-764b-4b82-b149-bc4d96950a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.166070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceb206-764b-4b82-b149-bc4d96950a83 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.167526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ceb206-764b-4b82-b149-bc4d96950a83 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:39.169069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ceb206-764b-4b82-b149-bc4d96950a83 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:39.175219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 212/430] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:38 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:39.175219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.175219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.175219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.189850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba40f4-ce28-43fc-87f6-06dd6c96dd90 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.190648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:39.192094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:39.231442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.232552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.232552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.232758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.233527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:39.233966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:39.235491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476af951-dfb6-4af8-85ff-e9576c1fbea3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:39.244346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 219/431] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=ee34d08904734afdb4939fd27d04b760 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:39.245181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.245471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.245750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.274950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 213/432] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:39.276261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.283176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:39.285902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:39.324848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:39.417221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 220/433] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:39.418010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.438590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b23255-fcfc-48dc-97f8-f093081f69a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.478717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:39.495999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.495999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.496455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.496615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.498780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:39.501073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:39.502707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 214/434] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:39.503378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.593013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4471b0-5b2f-4ac0-8adc-97e26070243b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.649355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.649355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.649355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.649719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.650708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bacc6-9a77-4fc9-a0b2-1039a33426d7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.663848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:39.673936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:39.675029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 221/435] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:39.675839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.706508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:39.712361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d425a2-115a-4100-89e3-57de98c42cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.712361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d425a2-115a-4100-89e3-57de98c42cbd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.712361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d425a2-115a-4100-89e3-57de98c42cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.712361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d425a2-115a-4100-89e3-57de98c42cbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.726244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d425a2-115a-4100-89e3-57de98c42cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b4d425a2-115a-4100-89e3-57de98c42cbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:39.734593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:39.735525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:39.736337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 215/436] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:39.740338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869737e-d3d7-465d-b902-7ea759d5f6cd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.740338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869737e-d3d7-465d-b902-7ea759d5f6cd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.740338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869737e-d3d7-465d-b902-7ea759d5f6cd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.744587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 216/437] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:39.744587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869737e-d3d7-465d-b902-7ea759d5f6cd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.808786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:39.810992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849d4c6-642b-4a46-9fa5-0a5e33a90d3a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.846346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.847158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.847450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.847819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.848818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=ee34d08904734afdb4939fd27d04b760, scope.project.id=d44454738f6541b6aa730195de6989a9) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:39.849363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:39.850843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:39.857003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 222/438] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=ee34d08904734afdb4939fd27d04b760 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:39.857801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:39.857962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869737e-d3d7-465d-b902-7ea759d5f6cd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:39.918267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:39.919086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.919377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.919662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.920552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:39.920867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:39.922510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c2d77-ba34-43fa-80fe-891a46e57cab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:39.938253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 217/439] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:39.939812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc493e66-877a-4f71-9da4-912889887071 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:39.944082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc493e66-877a-4f71-9da4-912889887071 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:39.944082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc493e66-877a-4f71-9da4-912889887071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:39.964800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cc493e66-877a-4f71-9da4-912889887071 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:39.966104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc493e66-877a-4f71-9da4-912889887071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:39.980890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.010549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f00c49-679b-483e-9737-49e853fc60db None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.052892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.053689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.053977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.054254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.055983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 218/440] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:40.056100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:40.056100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.056241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc493e66-877a-4f71-9da4-912889887071 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.056968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.072894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 223/441] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:40.075294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.121599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.182012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc493e66-877a-4f71-9da4-912889887071 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.202796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab895882-f747-40e8-9336-30f791abec2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.220587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.221259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.221544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.221826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.239033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.240029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.240527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.240890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.253755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.295331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.296972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.311220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.312718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.345248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.404101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 219/442] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:40.405023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.428288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 224/443] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:40.429179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.476233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.508327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ee2a-cdb6-463b-939a-471991ce1ec8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.536713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a7b4b-8574-4347-be2b-643c4977952e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.547062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.547875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.548167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.548495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.560777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:40.569541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:40.570592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:40.571419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 220/444] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:40.572090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.575872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.576811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.577094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.577370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] PATH_INFO: `/v3/projects/d44454738f6541b6aa730195de6989a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.578649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] RBAC: Authorizing `identity:get_project(project_id=d44454738f6541b6aa730195de6989a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.580123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.581889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 225/445] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/projects/d44454738f6541b6aa730195de6989a9 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:40.582551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.582731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.681440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74e8d1-9e3b-42de-b3c2-9f5a2c5a92b6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.705620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68860deb-879b-4e0d-8ead-6fe6088319d8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.717058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.718353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.718687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.718687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.719002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] PATH_INFO: `/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.720516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] RBAC: Authorizing `identity:get_project(project_id=b58b1b03bb4947b4be69f8dc27be3cb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.722016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.724039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 221/446] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/projects/b58b1b03bb4947b4be69f8dc27be3cb4 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:40.724958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.750481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.751289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.751638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.752004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.753348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:40.753728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.755841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.765277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 226/447] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:40.766161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.835649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:40.870554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506811d9-ea6f-445c-b0fd-4660f5a7a6bc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.876249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caad75-768a-4384-bc59-1773354ad744 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:40.907893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.908815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.909103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.909381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.911573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.912735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.917589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:40.918496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:40.918780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:40.919138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] PATH_INFO: `/v3/groups/39e50f0cb88b4dd5b6126200afbf5504` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:40.919555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 222/448] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:40.920463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] RBAC: Authorizing `identity:get_group(group_id=39e50f0cb88b4dd5b6126200afbf5504)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:40.920561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.921939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:40.923647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 227/449] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/groups/39e50f0cb88b4dd5b6126200afbf5504 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:40.924534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:40.974059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:41.032503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7722b-a772-4c82-81d3-7fb65206a956 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:41.033189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f5d2b-a8c4-4b5d-974f-3f1ac2e37aba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:41.068998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:41.069666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:41.069954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:41.070236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:41.071067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=9508980101c44da48df022abc45e1e60, scope.project.id=d44454738f6541b6aa730195de6989a9) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:41.071383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:41.071903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:41.072610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:41.072858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:41.072948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:41.073173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:41.080169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 228/450] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=d44454738f6541b6aa730195de6989a9&role.id=9508980101c44da48df022abc45e1e60 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:41.081758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:41.089833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:41.179326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:41.180882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:41.230908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbc349-9057-43d6-9b2f-e1f4b77953e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:41.233659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:41.273534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:41.274199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:41.275385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:41.275385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:41.275560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] RBAC: Adding query filter params (group.id=39e50f0cb88b4dd5b6126200afbf5504, role.id=9508980101c44da48df022abc45e1e60, scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:41.276483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:41.278114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:41.288029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 229/451] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:41 2022] GET /identity/v3/role_assignments?group.id=39e50f0cb88b4dd5b6126200afbf5504&scope.project.id=b58b1b03bb4947b4be69f8dc27be3cb4&role.id=9508980101c44da48df022abc45e1e60 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:41.317128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 223/452] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:41.322869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:41.439969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b280541-46b2-4765-98a8-8738ca8fbdf3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:41.481220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:41.482007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:41.482288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:41.482566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] PATH_INFO: `/v3/projects/56278f94460f422387fbeb96048d2270` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:41.483772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] RBAC: Authorizing `identity:get_project(project_id=56278f94460f422387fbeb96048d2270)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:41.485277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:41.487777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 230/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:30:41 2022] GET /identity/v3/projects/56278f94460f422387fbeb96048d2270 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:41.494230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:41.549283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:41.616857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27854f52-bf7c-44aa-8ef9-25e95fbbd433 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:41.653700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a377-550f-48f4-9d4f-750d57ec867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:41.654556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a377-550f-48f4-9d4f-750d57ec867c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:41.654840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a377-550f-48f4-9d4f-750d57ec867c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:41.655120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a377-550f-48f4-9d4f-750d57ec867c None admin] PATH_INFO: `/v3/groups/8d60acee023f4252a333d2df06622c5b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:41.656348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0a377-550f-48f4-9d4f-750d57ec867c None admin] RBAC: Authorizing `identity:get_group(group_id=8d60acee023f4252a333d2df06622c5b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:41.657805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0a377-550f-48f4-9d4f-750d57ec867c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:41.659436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 224/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu May 26 13:30:41 2022] GET /identity/v3/groups/8d60acee023f4252a333d2df06622c5b => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:41.665982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:41.712754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:41.787420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1ad8b-2eeb-4510-98b1-21c685258bbd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:41.824248 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:41.824943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:41.825236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:41.825514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:41.826353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] RBAC: Adding query filter params (group.id=8d60acee023f4252a333d2df06622c5b, role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=56278f94460f422387fbeb96048d2270) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:41.826672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:41.828120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091de813-7856-4775-a3c0-b47bdbe199d0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:41.834586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 231/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu May 26 13:30:41 2022] GET /identity/v3/role_assignments?group.id=8d60acee023f4252a333d2df06622c5b&scope.project.id=56278f94460f422387fbeb96048d2270&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:44.904279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11d70-c571-4fe1-bc06-36b068c733c3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:44.904279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11d70-c571-4fe1-bc06-36b068c733c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:44.904279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11d70-c571-4fe1-bc06-36b068c733c3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:44.905549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 225/456] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:44.924212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc38db-5845-46fd-bdee-2718cf700c98 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:44.924212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc38db-5845-46fd-bdee-2718cf700c98 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:44.924212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc38db-5845-46fd-bdee-2718cf700c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:44.924817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650ad09-0b16-4f3d-be74-d3fd38e44a19 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:44.924817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650ad09-0b16-4f3d-be74-d3fd38e44a19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:44.924817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650ad09-0b16-4f3d-be74-d3fd38e44a19 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:44.929471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 226/457] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:30:44.931020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:44.931505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:44.931974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:44.960335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-02fc38db-5845-46fd-bdee-2718cf700c98 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:44.960335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02fc38db-5845-46fd-bdee-2718cf700c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:44.982694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:44.986172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:45.098066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 232/458] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:45.104344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968942ef-f503-4b18-ad4a-aa3ca99f1dd0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.104344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968942ef-f503-4b18-ad4a-aa3ca99f1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.104344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968942ef-f503-4b18-ad4a-aa3ca99f1dd0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.109692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 233/459] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:45.117524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.117801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.118078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.141315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:45.142492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:45.143400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 227/460] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:45.144262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:45.200407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:45.212743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:45.274013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c826c-213e-4c46-a81f-e5431928f452 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:45.319737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:45.320834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.321573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.321988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.325080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 234/461] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:30:45.326215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:45.335307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:45.337476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:45.366999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:45.368135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:45.369342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:45.369342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:45.369342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:45.369342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:45.372140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 228/462] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:45.374236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:45.496905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:45.510891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f129203-2210-4846-87ef-0678ce0f0d8e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:45.520788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066456-089a-41d4-bc30-1c51a0590b7e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:45.560325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:45.561919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.562627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.563118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.576984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:45.578936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:45.603814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:45.604966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:45.605823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:45.605823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:45.605823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:45.605823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:45.608265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 229/463] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:45.609948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:45.645176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:45.646027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.646321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.646607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.681382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:45.682890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:45.736094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c418f35-59ed-4aee-8cf9-9f9b9c7864f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:45.741360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:45.742409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:45.743275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:45.743275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:45.743275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:45.743275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:45.753418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 235/464] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:45.755153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:45.793673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:45.796791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:45.797590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.798317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.798317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.799055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:45.799426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:45.801553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:45.834136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 230/465] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:45.835214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:45.846244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:45.919622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c7164-d3fe-44a6-80c7-6db1f58632fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:45.965847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:45.967566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:45.968202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:45.968751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:45.969745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:45.970247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:45.972104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:45.985423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 236/466] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:45.986789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:46.029111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d61315-c9cf-489d-a305-b8755df98033 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:46.073360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:46.086387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:46.087502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.087971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.088296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.089061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:46.089369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:46.090773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98040d-aa18-4093-8300-9205bdbda093 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:46.109780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 231/467] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:46.109780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37f7f0-ee06-4c99-9e48-4278aaf461d4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.109780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37f7f0-ee06-4c99-9e48-4278aaf461d4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.109780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37f7f0-ee06-4c99-9e48-4278aaf461d4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.109780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 232/468] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:46.110210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37f7f0-ee06-4c99-9e48-4278aaf461d4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:46.123417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5113c-64c6-4705-8743-efbddee07308 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:46.173860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:46.174879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.175169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.175518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.177069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:46.179379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:46.194708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:46.216577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 237/469] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:45 2022] POST /identity/v3/users => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:46.216577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:46.253145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37f7f0-ee06-4c99-9e48-4278aaf461d4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:46.325593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8b4e4-2a4f-4708-9dbe-77e034b8d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:46.326274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8b4e4-2a4f-4708-9dbe-77e034b8d5c9 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.326677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8b4e4-2a4f-4708-9dbe-77e034b8d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.327024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8b4e4-2a4f-4708-9dbe-77e034b8d5c9 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.327823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8b4e4-2a4f-4708-9dbe-77e034b8d5c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:46.329591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8b4e4-2a4f-4708-9dbe-77e034b8d5c9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:46.360101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 233/470] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:46 2022] POST /identity/v3/users => generated 248 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:46.361899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b62dcb-1904-49ca-8e3f-633925b76d79 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.362198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b62dcb-1904-49ca-8e3f-633925b76d79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.362480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b62dcb-1904-49ca-8e3f-633925b76d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.386973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-77b62dcb-1904-49ca-8e3f-633925b76d79 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:46.388457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-77b62dcb-1904-49ca-8e3f-633925b76d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:46.399146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40d76-4e51-4f58-b6cf-fe2f770ce7e6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:46.456765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:46.481324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53ba56-1642-4176-b3ec-4cc507cc8872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:46.482683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53ba56-1642-4176-b3ec-4cc507cc8872 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.483755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53ba56-1642-4176-b3ec-4cc507cc8872 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.484232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53ba56-1642-4176-b3ec-4cc507cc8872 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.487580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53ba56-1642-4176-b3ec-4cc507cc8872 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:46.490559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53ba56-1642-4176-b3ec-4cc507cc8872 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:46.524203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 238/471] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:46 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:46.527379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 234/472] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:46.533735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7656d-f1d3-4f29-9cc6-ff5e2eb37763 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.534322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7656d-f1d3-4f29-9cc6-ff5e2eb37763 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.534791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7656d-f1d3-4f29-9cc6-ff5e2eb37763 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.536995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 239/473] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:46.543597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.543891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.544408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.561581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7656d-f1d3-4f29-9cc6-ff5e2eb37763 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:46.580176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:46.581302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:46.665170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:46.698417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 235/474] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:46.725016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:46.755001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7656d-f1d3-4f29-9cc6-ff5e2eb37763 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:46.797599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:46.801053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.801641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.802170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.815603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:46.818355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:46.843809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:46.844775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:46.845627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:46.845627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:46.845627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:46.845627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:46.847702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 240/475] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:46.854924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:46.876501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c142f-fa7a-4d59-8157-6f37cd9e9142 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:46.924272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:46.925626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:46.926203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:46.927070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:46.928751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:46.929908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:46.932545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:46.943994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 13:30:46.943994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 13:30:46.943994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [parameters: {'id': '85855db990de4c268ca2629f5850050a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 13:30:46.943994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 26 13:30:46.968753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return ref.to_dict() May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:30:46.969717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 13:30:46.970660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 13:30:46.971602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:30:46.973347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:30:46.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': '85855db990de4c268ca2629f5850050a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:46.978206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 13:30:46.980184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:46.985341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 236/476] 10.222.0.16 () {60 vars in 1131 bytes} [Thu May 26 13:30:46 2022] POST /identity/v3/roles => generated 144 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:46.994404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:47.075721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4baa6-382b-4a96-837f-cb3ff5b4fdd0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:47.078276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:47.143702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:47.145700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:47.146801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:47.147438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:47.148790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:47.149318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:47.151741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:47.177125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 241/477] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:47.185817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:47.202354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b9e86-5920-4cfa-86b2-535d67a6ab87 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:47.302546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:47.304336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:47.304336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:47.304336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:47.349706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:47.379454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:47.380451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 13:30:47.381261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:47.384908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 237/478] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:30:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:47.390627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:47.452502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:47.480345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c08ef-4337-4e51-87d0-d36a58064082 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:47.559494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed639365-a89e-433e-897c-3925c94d06a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:47.560190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed639365-a89e-433e-897c-3925c94d06a0 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:47.564395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed639365-a89e-433e-897c-3925c94d06a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:47.564395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed639365-a89e-433e-897c-3925c94d06a0 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:47.564395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed639365-a89e-433e-897c-3925c94d06a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:47.564395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed639365-a89e-433e-897c-3925c94d06a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:47.611098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 242/479] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:47 2022] POST /identity/v3/users => generated 249 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:47.638792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7c81-d235-4e7b-a2d6-a19794c96fe0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:47.676343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:47.695740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:47.696475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:47.696769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:47.697053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:47.697827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:47.698142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:47.699578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd36f6-6055-4ee4-b3f7-cee91f0b5be0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:47.715561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 238/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:49.173073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83272cd-66a3-4542-8a75-3b7fe8859449 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:49.173550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83272cd-66a3-4542-8a75-3b7fe8859449 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:49.173550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83272cd-66a3-4542-8a75-3b7fe8859449 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:49.175241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 243/481] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:49.180809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615eaba4-98d5-4dd1-8d87-ffb6f18c2b4c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:49.181287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615eaba4-98d5-4dd1-8d87-ffb6f18c2b4c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:49.181361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615eaba4-98d5-4dd1-8d87-ffb6f18c2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:49.209703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-615eaba4-98d5-4dd1-8d87-ffb6f18c2b4c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:49.211064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-615eaba4-98d5-4dd1-8d87-ffb6f18c2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:49.330623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 239/482] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:49.367761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83272cd-66a3-4542-8a75-3b7fe8859449 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:49.605098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83272cd-66a3-4542-8a75-3b7fe8859449 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:49.633369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4876a-e7d9-416b-9c6c-41bb7058d3f0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:49.633720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4876a-e7d9-416b-9c6c-41bb7058d3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:49.635284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4876a-e7d9-416b-9c6c-41bb7058d3f0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:49.635400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 240/483] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:49.644345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:49.644345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:49.644345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:49.675151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:49.679244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:49.683916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:49.683916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:49.683916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:49.683916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:49.705857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:49.719081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:49.720090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:49.721026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:49.725705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 244/484] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:49.733889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:49.774033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:49.846717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 241/485] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:49.890930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:49.957212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:49.963659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced73d4-cd3b-4950-8ee3-07c6e7c93a2d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:50.059945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:50.060820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.064347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.064347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.064347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:50.064347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:50.064347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:50.088079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 245/486] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:50.098240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:50.156414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df368ca-fe71-4262-b4dc-bc9f8fa4ed6a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:50.227221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f10e2a-f74d-4289-9cee-e3551ad282e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:50.228032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f10e2a-f74d-4289-9cee-e3551ad282e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.228391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f10e2a-f74d-4289-9cee-e3551ad282e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.228672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f10e2a-f74d-4289-9cee-e3551ad282e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.230219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:50.254043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f10e2a-f74d-4289-9cee-e3551ad282e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:50.263219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4bbf9-85b9-45b3-9945-ed5656ff21ca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c1f10e2a-f74d-4289-9cee-e3551ad282e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:50.266883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:50.267827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:50.268688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 242/487] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:50.270992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e2913-0f84-4742-a69f-ee8639cb4b50 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.271282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e2913-0f84-4742-a69f-ee8639cb4b50 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.271568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e2913-0f84-4742-a69f-ee8639cb4b50 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.276369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 243/488] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:50.282044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e2913-0f84-4742-a69f-ee8639cb4b50 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:50.323433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:50.326821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.327592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.329675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.355394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:50.370082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a5f350c7-9327-46c7-8a31-1f193a47acd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:50.414392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:50.415300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:50.416160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:50.416160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:50.416160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:50.416160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:50.420255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 246/489] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:50.422992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.425296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.426592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.469302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:50.470876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:50.499036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:50.528144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e2913-0f84-4742-a69f-ee8639cb4b50 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:50.603809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:50.605016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.605502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.608546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.612168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:50.612534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:50.618417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:50.651056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 244/490] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:50.654775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:50.676413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 247/491] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:50.676413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:50.757355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:50.821349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2b44-1f54-4f14-b990-92bf94a0d842 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:50.852946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aa58-4722-4b85-a202-64a968d111d5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:50.866292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:50.867012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.867301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.867578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.881042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:50.883221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:50.905382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:50.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:50.907168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:50.907168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:50.907168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:50.907168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:50.911522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 248/492] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:50.914617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:50.915619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:50.916105 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:50.916606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:50.917552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:50.918066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:50.922181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:50.930922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:50.948570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 245/493] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:50.949823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:50.970633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.093868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40f020-50f2-45e3-b72e-d074aeb9dd47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.105679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729b6a4-0a35-4ff4-a49b-c8d970bbb93d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.107291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.146221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe0e94-0c90-409f-9474-6fdf0b73e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.147336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe0e94-0c90-409f-9474-6fdf0b73e2cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.147783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe0e94-0c90-409f-9474-6fdf0b73e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.150474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe0e94-0c90-409f-9474-6fdf0b73e2cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.151453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.152281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.152482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.152824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.153427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:51.153735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:51.155159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:51.164784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe0e94-0c90-409f-9474-6fdf0b73e2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:51.172516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 246/494] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:51.173507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9bfe0e94-0c90-409f-9474-6fdf0b73e2cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:51.173733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:51.174842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:51.177221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 249/495] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:51.178622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b401be9-6ada-4b2c-9b7f-12bfe2ab456d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.178922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b401be9-6ada-4b2c-9b7f-12bfe2ab456d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.179229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b401be9-6ada-4b2c-9b7f-12bfe2ab456d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.181809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 250/496] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:51.182983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e261d-18b5-458e-9d30-72b83eae203c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.183273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e261d-18b5-458e-9d30-72b83eae203c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.183556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e261d-18b5-458e-9d30-72b83eae203c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.189316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 251/497] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:51.189316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e261d-18b5-458e-9d30-72b83eae203c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.261956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.303944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa9ac2-5a92-4c18-ac81-eaaf8807c625 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.343557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e261d-18b5-458e-9d30-72b83eae203c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.351129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.352098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.352483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.352778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.366007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:51.367561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:51.384884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.385687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.385975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:30:51.386225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:51.387149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:30:51.388080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:51.388680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.388752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 247/498] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:51.388752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.397482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:51.398967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1ae6fd58-d4b5-4ab6-b674-fbe7debe192c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:30:51.418587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:51.419786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:51.420733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 252/499] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:51.421711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783a71a-1c90-4828-badf-9d8c5934df37 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.422000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783a71a-1c90-4828-badf-9d8c5934df37 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.422276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783a71a-1c90-4828-badf-9d8c5934df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.440555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b783a71a-1c90-4828-badf-9d8c5934df37 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:51.442561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b783a71a-1c90-4828-badf-9d8c5934df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:51.477627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.527440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca4445-5062-494b-a637-2a29bcccfef2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.552089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 253/500] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:51.552089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.552089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.552089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.565211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:51.566870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:51.573468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.583373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.584323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.584619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.584901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.585713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:51.586161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:51.588126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:51.597044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 248/501] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:51.598447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.658055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 254/502] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:51.658751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.727431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b356fa-b2a6-4b8c-96ef-40bcc78b5009 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.772766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.773768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.774053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.774331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.775201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:51.775865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.776035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:51.777714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:51.800546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e0d87-67ad-4383-9524-d74d23b45dd0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:51.823678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 249/503] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:51.824788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.848321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:51.849238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:51.849524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:51.849898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:51.850663 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:51.850974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:51.852701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:51.900339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:51.903795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 255/504] 10.222.0.16 () {56 vars in 1137 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:51.905463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:51.972106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e1a8-0bfa-4335-96aa-f04b0270c1a6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.012733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.013505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.013790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.014068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.027058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:52.035592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:52.036682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:52.037651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:52.038843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 250/505] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:52.039747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.053074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92568888-1a5a-4e15-b75b-ee53349002df None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.095242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.096443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.096729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.097002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.111470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.113090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.121312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:52.135853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:52.136850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:52.138358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:52.138358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:52.138358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:52.138358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:52.138358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 256/506] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:52.138358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.180441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe5124-c89b-4760-b45f-f1ad9b97f30f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.192855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:52.230132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.231006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.231296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.231582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.246031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.247066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:52.270502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:52.271782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:52.273150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:52.273150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:52.273150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:52.273150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:52.273150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 251/507] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:52.273150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.276765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d8905-f2fe-407d-b8ac-6e8bcd4c0070 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.327583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.328951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.329381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.329788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.412373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e1656-1467-45e7-a1ad-e6b78c647ff7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.413740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:52.420659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.422698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.462390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.463560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.463753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.464164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.500282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:52.554154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 257/508] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:52.554154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.555361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.558546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.676684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:52.681117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe42ff-9deb-41a0-8651-4202cc7f52fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.682212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 252/509] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:52.683154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.731844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.732553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.732844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.733122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.734169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:52.734477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.736441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.747314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 258/510] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:52.752732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.781315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:52.843206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaad011-f631-47aa-bf35-b3646a233858 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.888106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e1b2b-7853-4f57-b07c-3fc3f6c788c4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:52.891999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.892794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.893086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.893406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.894699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:52.895021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.897269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.919068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 253/511] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:52.919807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.934503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590451af-a10d-4279-897a-041ad1651777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:52.935536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590451af-a10d-4279-897a-041ad1651777 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:52.936504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590451af-a10d-4279-897a-041ad1651777 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:52.936857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590451af-a10d-4279-897a-041ad1651777 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:52.937824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590451af-a10d-4279-897a-041ad1651777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:52.938154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590451af-a10d-4279-897a-041ad1651777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:52.939982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590451af-a10d-4279-897a-041ad1651777 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:52.953749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 259/512] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:52.959232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590451af-a10d-4279-897a-041ad1651777 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:52.987652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:53.056568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84183197-9ae4-4372-a679-5ff57ac158bf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.094398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:53.121370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.122752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.123036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.123463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.125422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590451af-a10d-4279-897a-041ad1651777 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.141600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.144541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.165784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.166952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.167275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.167555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:53.172556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:53.173448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:53.174310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:53.174310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:53.174310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:53.174310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:53.174310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 254/513] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:53.175006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.180463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.182088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:53.202385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:53.204023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:53.205553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:53.205553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:53.205553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:53.205553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:53.205553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 260/514] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:53.205553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.306618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:53.330483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d03c82-b727-4f29-aa24-038b8b888202 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.342053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db2f4-bfa3-47d0-9f69-cfd8aea9404d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.379669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.380606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.381095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.381626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.385689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.386509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.386788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.387063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.394942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.396523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.397295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.398878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:30:53.416187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:53.417753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 261/515] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:53.418682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:53.424266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:53.425359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:53.426245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:53.426245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:53.426245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:53.426245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:53.426245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 255/516] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:53.426842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.466168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:53.558597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800d1fe-ddb7-4f85-aa2c-e636b2b29632 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.578754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d343-b7ec-4f3c-bbdf-d9af0bb4a15f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.598301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:53.604973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.605667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.605974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.606256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.607152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.608739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.622462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.623418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.623773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.624022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.624855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:53.625316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.626759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.633402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 262/517] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:30:53 2022] POST /identity/v3/users => generated 247 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:30:53.634243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.642195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 256/518] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:53.642717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.775678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:53.806355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c70e42-4284-4e7f-aa1b-fc0e95d3d74a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.812487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f418-f277-4646-ba44-f6e979072640 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:53.873039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680823-c478-4b31-8a26-db8774601971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.873405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:53.874032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680823-c478-4b31-8a26-db8774601971 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.874330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680823-c478-4b31-8a26-db8774601971 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.874558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:53.874681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680823-c478-4b31-8a26-db8774601971 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.875247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:53.875464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52680823-c478-4b31-8a26-db8774601971 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:53.875571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:53.875779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52680823-c478-4b31-8a26-db8774601971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.876241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:53.876584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:53.878189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52680823-c478-4b31-8a26-db8774601971 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.878358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:53.898426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 263/519] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:53.900379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.914757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 257/520] 10.222.0.16 () {56 vars in 1137 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:53.914757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680823-c478-4b31-8a26-db8774601971 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:53.980840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.058108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5feadc-bde2-4c31-96cb-87b805a27f07 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.070346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680823-c478-4b31-8a26-db8774601971 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.104946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.105740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.106537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.106790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.107576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:54.107910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.109643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.113877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.114950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.115380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.116018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.124522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 264/521] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:54.124857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.126812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.129345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:30:54.147764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:54.148906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:54.149895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:54.156943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 258/522] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:54.158340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.162030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.306057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe177874-7dce-4eb4-9728-f2b34d6ff06c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.313291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd299fd6-46f0-405a-81fd-4310d0f27b52 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.316330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.345488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.346314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.346596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.346870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.353715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.354424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.354715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.354939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.356122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.357664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:30:54.375136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:54.376075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:54.377021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 265/523] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:54.378359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.483114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.484850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.487271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.493983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb739708-b3d2-4d29-b40c-5778bc3139fd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.531885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.532743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.533034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.533317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.545518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.547395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:30:54.572768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:54.573912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:30:54.575044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:54.582556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 266/524] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:54.583854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.632592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.638818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 259/525] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:54.640068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.745188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90295-7f4b-4239-927c-e43c4288d38a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.778005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a956d3-4c26-437e-88a1-7848496035c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:54.781023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.802302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.803126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.803419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.803700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.804691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:54.804875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.806292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.828829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:54.829534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:54.829820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:54.830098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:54.837003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 267/526] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:54.837185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:54.837185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:54.837337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.837410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:54.857546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 260/527] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:54.859012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:54.965976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:54.985892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfa2ee-1cb9-4981-a5cf-de4fe2b90d9b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:55.015059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79f355-cc98-472f-bfb8-1f9334e39000 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:55.037059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:55.037759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:55.038044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:55.038327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:55.039189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:55.039510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:55.041047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:55.065545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 268/528] 10.222.0.16 () {56 vars in 1139 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:55.066685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:55.080227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:55.081599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:55.082106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:55.082778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:55.102785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:55.105270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:55.135732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:55.136697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:55.137544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:55.137544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:55.137544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:55.137544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:55.145949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 261/529] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:55.146638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:55.172910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:55.239481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c4d0-ecbc-4ab2-915d-09848ac50cc2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:55.297797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:55.298440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:55.298681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:55.298970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:55.299810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=535f23db24704422a4128ff09f1cc284) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:55.300159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:55.301677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:55.305145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cfed5-c291-42af-868c-773d1d7cc4dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:55.317165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 269/530] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:55 2022] GET /identity/v3/role_assignments?user.id=535f23db24704422a4128ff09f1cc284&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:55.318528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:55.367498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:55.369989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:55.373543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:55.374046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:55.377288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:55.378188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=867e9b57fa7947e3afcbd974ed887969) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:55.379279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:55.380556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:55.396349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 262/531] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:55 2022] GET /identity/v3/role_assignments?user.id=867e9b57fa7947e3afcbd974ed887969&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:55.396349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:55.577047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:55.609694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293f335-6a92-4481-80bd-afe505c90ff4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:55.669329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:55.670526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:55.670824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:55.671921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:55.674342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:55.674694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:55.677762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:55.696528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 263/532] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:55.702086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:55.754756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc4e2-1309-4f66-826a-cfb0eb126a08 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:55.885465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac485fc-c07c-4dea-bfb3-ba2a1f409bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:55.891831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac485fc-c07c-4dea-bfb3-ba2a1f409bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:55.892456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac485fc-c07c-4dea-bfb3-ba2a1f409bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:55.893647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac485fc-c07c-4dea-bfb3-ba2a1f409bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:55.959746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf61eb-734a-496d-8947-38e8c3beedae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:56.104620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac485fc-c07c-4dea-bfb3-ba2a1f409bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:56.106133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac485fc-c07c-4dea-bfb3-ba2a1f409bb4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:56.122112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:56.123323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:56.123813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:56.124291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:56.136804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:56.146648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:56.148162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:56.150615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5bdc29bc-9db8-4995-90ef-3a1b2a9ac403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:30:56.182373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:56.183296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 264/533] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c47ab-e065-4c00-b7ac-2579f0de4568 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:56.184165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c47ab-e065-4c00-b7ac-2579f0de4568 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:56.184792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c47ab-e065-4c00-b7ac-2579f0de4568 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:56.197302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 265/534] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:56.199393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c47ab-e065-4c00-b7ac-2579f0de4568 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:56.276336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 270/535] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 947 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:56.276336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:56.276336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:56.276336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:56.300591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:56.304186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:56.444332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 271/536] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:56.444332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:56.540340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:56.592340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c47ab-e065-4c00-b7ac-2579f0de4568 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:56.632821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ac842-a130-4793-8494-51c4540cd35a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:56.689079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:56.690135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:56.690620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:56.691887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:56.693050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:56.693583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:56.695331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:56.710410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 266/537] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:56.711816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:56.725073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:56.726273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:56.726758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:56.727230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:56.743433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:56.745633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:56.779130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:56.780820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:56.781681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:56.781681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:56.781681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:56.781681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:56.797852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 272/538] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:56.800158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:56.876856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:57.066651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a68563-e4ab-46e0-b32c-b00e7680d3a3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:57.207918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:57.250238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03165d5-6db3-4df5-b743-4d1df9dc46be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:57.257074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:57.258897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.259411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.259891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.286132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:57.296767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:57.297695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:57.298492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:57.301330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:57.306723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.306723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.306723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.306723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=4e8552f561ce4b4fa837f907a58816aa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:57.306723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:57.306723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:57.307471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 267/539] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 591 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:57.308821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:57.312171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 273/540] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:56 2022] GET /identity/v3/role_assignments?user.id=4e8552f561ce4b4fa837f907a58816aa&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:57.313099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:57.480693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b368aee-e89d-4f13-89e7-ef8e545583d8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:57.510464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998deac-560d-4fa5-8e44-692996b6c4f0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:57.522617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:57.523581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.524142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.524601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.526471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:57.527039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:57.528958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:57.540542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 274/541] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:57.546345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:57.559299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:57.572661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:57.598262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:57.598947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.599235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.599519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.600254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:57.600596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:57.601995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:57.618778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 268/542] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:57.623802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:57.722087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffa27f-f413-4bc8-b36b-b17da49933fc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:57.793675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:57.795179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.795692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.796179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.809431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:57.811283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:57.834310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:57.835226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:57.836051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:57.836051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:57.836051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:57.836051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:57.842562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 275/543] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:57.844669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba75ad7-0f88-4f40-8e12-9baa8d2ee5ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:57.847353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:57.907242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:57.908598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.920407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.920407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.930793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:57.935555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a4b68dfa-700f-496f-8f4d-7585bf471e3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:30:57.953037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:57.954129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:57.955270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 269/544] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:30:57.955907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9f96-a419-4d02-8176-bb7c0b76fd65 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:57.957497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9f96-a419-4d02-8176-bb7c0b76fd65 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:57.957497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9f96-a419-4d02-8176-bb7c0b76fd65 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:57.958186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 270/545] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:57.962962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9f96-a419-4d02-8176-bb7c0b76fd65 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:57.976670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.010103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.029345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86228b-f142-4582-9d29-b683fb81988d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.085214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9f96-a419-4d02-8176-bb7c0b76fd65 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.086170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.087093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.088783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.089117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.089973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:58.090388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:58.091775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59546749-58fb-4fa7-9678-ec326ad2bb7d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:58.109002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 276/546] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:58.109002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.109002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.109002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.125312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:58.126614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:58.127183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.127857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.128736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.129154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.130183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:58.130497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:58.131979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb55e9e-1ed8-4ff8-9ffa-f74154e4db3c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:58.151638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 271/547] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:58.153063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1919f2-907a-473c-ac94-8f52ea988ce9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.153353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1919f2-907a-473c-ac94-8f52ea988ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.153641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1919f2-907a-473c-ac94-8f52ea988ce9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.155118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 272/548] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:58.155640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1919f2-907a-473c-ac94-8f52ea988ce9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.222387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 277/549] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:58.222387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.272535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.277203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1919f2-907a-473c-ac94-8f52ea988ce9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.310824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.330217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.332225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.332842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.333222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.344156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:58.345710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:58.362118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7b24f-1a8a-41c5-bffb-02b74fd0a157 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b8ab1907-19c1-4953-bf99-329ae6fcd131 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:30:58.374377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:58.375383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:30:58.376401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:58.380628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 273/550] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:58.382723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.382723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.382896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.404417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:30:58.405640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:30:58.415021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.415967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.416861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.417322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.418903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=6f44d10be1654d17990966ae64d43ebd) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:58.419528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:58.421971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:58.429271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 278/551] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/role_assignments?user.id=6f44d10be1654d17990966ae64d43ebd&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:58.430299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.549842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f30b5-c863-4295-b348-35705f727f36 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.588250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.596563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 274/552] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:30:58.596563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.626810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.628010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.628888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.629580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.643430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:58.652224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:58.653424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:58.654231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:58.656432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 279/553] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:58.657855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.730255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.744123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4ec3-9939-4fbf-90cc-cae4c5826f1d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.786870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.787563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.787849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.788129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.789011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:58.789324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:58.790808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:58.808432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e2a2d-b9cc-4975-9b57-4fa59e48c4af None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.814727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 275/554] 10.222.0.16 () {56 vars in 1135 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:58.815747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.857769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.858499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.858782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.859059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.873874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:30:58.886179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:58.887211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:58.888020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 280/555] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:58.888737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:58.941991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a999e-8789-4476-b6f5-5dde8d039153 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:58.947629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:58.983306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:58.984394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:58.984825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:58.985338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:58.986291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:58.986929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:58.989093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.003908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 276/556] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:59.005258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.007304 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514b997-9fd6-4a65-9b8e-5163a6d04776 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.017626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:59.043400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.044113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.044474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.044729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.120730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.122062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.123345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263641ef-e2eb-4c81-9987-20595a2960f1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.164113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.165304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.165825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.166325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.167408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:59.167944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.169705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.171856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:59.187320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 277/557] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:59.188841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.244112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 281/558] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:30:59.245150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.297955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:59.345561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cdafe-1da1-4bbd-a01a-0cc41814d094 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.406372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.408269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.408736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.409183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.410814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6f1f5-56d8-4a68-bc33-e7e0a730e3b6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.426032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.428729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:59.452383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:59.453528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:59.454642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:59.454642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:59.454642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:59.454642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:59.461693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 278/559] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:59.461693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.464550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.465758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.466050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.466331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.481264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.483675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.488056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:59.521492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:59.523344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:59.525201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:59.525201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:59.525201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:59.525201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:30:59.534805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 282/560] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:59.536075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.588553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c9e2-3af5-4dfa-8b90-550b2a07bf2d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.623791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.624703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.624990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.625272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.639595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.641243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.642358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:59.654695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652cad5-c23c-45ea-a581-e9f47f52a8ed None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:30:59.667638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:30:59.668618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:30:59.669588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:30:59.669588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:30:59.669588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:30:59.669588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:30:59.670795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 279/561] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:30:59.671926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.705193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.705877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.706194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.707001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.707743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:59.708056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.709785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.729133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 283/562] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:59.729133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.793956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:30:59.826417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d7178-3871-47c6-99c3-2873dfdaa34c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.855733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d415f6-a004-402f-a3c2-59e1af48a529 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:30:59.867859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.868551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.868833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.869114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.869830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:59.870135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.871522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.884771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 280/563] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:59.885620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.901543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:30:59.902216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.902507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.902792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.903519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:30:59.903837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:30:59.905289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947d6b-d8d4-4d1a-bdef-f2445b7d8104 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:30:59.926247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 284/564] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:30:59.927571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5fb3-cc2f-485c-b338-d45fd110a920 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:30:59.927861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5fb3-cc2f-485c-b338-d45fd110a920 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:30:59.928176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5fb3-cc2f-485c-b338-d45fd110a920 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:30:59.932477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 285/565] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:30:59.932477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5fb3-cc2f-485c-b338-d45fd110a920 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:30:59.939836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.028828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287819d-0765-41b2-8773-da2ccc2ef027 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.039706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5fb3-cc2f-485c-b338-d45fd110a920 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.057358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.070281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.071161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.071453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.071717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.078555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.079350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.079561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.079664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.079899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.081036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.088176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.089664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:00.097900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:00.099197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:00.100193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 281/566] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:00.100842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a748f05c-917b-4e27-9513-63e55f5a18a6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:00.110278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:00.111257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 286/567] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:30:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.119280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.121771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.133014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:00.134304 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:00.207102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 287/568] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:00.211388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.214532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.221235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d2a01-2104-4b65-a832-bad9c2a9cfa3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.260339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.261140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.261428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.261704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.269383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.270918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:00.284870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:00.285811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:00.286751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 282/569] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:00.287255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.336036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739c5c-cd33-47bc-9689-527a9b1f44fa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.344093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.374025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.374731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.375440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.375778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.376661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:00.377330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.379659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.398149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 288/570] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:00.399146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.406155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970d4e9-1844-4de3-bddf-0a5a181cf63e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.446375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.447080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.447761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.448047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.449196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:00.449509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.451727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.475533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 283/571] 10.222.0.16 () {56 vars in 1137 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:00.476541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.492277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.516261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4ffc-1976-4121-a801-e5d1e0d0cc6d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.557892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.558717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.559006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.559301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.571412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:00.587793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:00.589061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:00.590117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:00.590783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 289/572] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:00.591945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.598819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682deb0-3a4d-4729-8809-556f0c9359ec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.627179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.643259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.644090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.644415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.644703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.645502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:00.645815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.647295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.661748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 284/573] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:00.662433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.722795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae947-81e3-4515-8fce-8df4208c7550 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.748012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.761300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.762113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.762400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.762681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.779509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.781104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.797989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca865-2c86-41aa-8919-b5ecd8d0d7e3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:00.801067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:00.802144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:00.803355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:00.803355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:00.803355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:00.803355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:00.803355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 290/574] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:00.803355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.834042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc626b0c-8683-4448-9856-575361734534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.834879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc626b0c-8683-4448-9856-575361734534 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.835156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc626b0c-8683-4448-9856-575361734534 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.835434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc626b0c-8683-4448-9856-575361734534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.846586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc626b0c-8683-4448-9856-575361734534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.848554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc626b0c-8683-4448-9856-575361734534 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bc626b0c-8683-4448-9856-575361734534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:00.869068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:00.870213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:00.871309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:00.871309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:00.871309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:00.871309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:00.871309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 285/575] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:00.871644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc626b0c-8683-4448-9856-575361734534 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:00.904044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:00.932213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3005a-ff9e-4583-bb63-647b643e552b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.969102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:00.969782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.970075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.970359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.971110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:00.971398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:00.973534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e9555-79c0-4839-9b86-6cc7f32a1719 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:00.982362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 291/576] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:00.983699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc1e60-2905-4648-ba76-07b6788a0a8d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:00.983988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc1e60-2905-4648-ba76-07b6788a0a8d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:00.984265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc1e60-2905-4648-ba76-07b6788a0a8d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:00.984956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc626b0c-8683-4448-9856-575361734534 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:00.985581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 292/577] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:00.986300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc1e60-2905-4648-ba76-07b6788a0a8d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.013052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.026682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.027336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.027639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.027980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.028907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=e4e2c8a980714257b8df1d486d446eeb) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:01.029223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.030714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.039139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 286/578] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/role_assignments?user.id=e4e2c8a980714257b8df1d486d446eeb&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:01.040003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.133369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc1e60-2905-4648-ba76-07b6788a0a8d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.178509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.179506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.179893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.180281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.181182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:01.181609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.183328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.193459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.194157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846c380-13ad-4fab-81ff-c8768c9f52a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.200742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 293/579] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:01.202223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.244531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.246121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.246121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.246121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.246564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:01.246747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.248163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a91f62-c1e7-47b4-abb5-f4a9477333f8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.266790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 287/580] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:01.267833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.268122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.268494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.287481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:01.288734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:01.350172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa16581-0c9e-4587-8864-2865e160b521 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.351929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.385192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 288/581] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:01.386066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.417363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.418277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.418673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.418952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.431846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.433495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:01.461328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:01.462360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:01.463245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:01.463245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:01.463245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:01.463245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:01.463245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 294/582] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:01.463515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.512857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2be455-634a-48e8-b5f3-d7bd0298f67e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.526682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.553821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.554627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.554906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.555176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.562738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.564362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:01.579351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:01.580350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:01.582739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 289/583] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:01.583228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.590781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbb5a-3c26-4841-b5ad-7942bba277f0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.627186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.636319 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.637166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.637424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.637707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.646437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.648255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:01.662695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:01.664026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 295/584] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:01.664967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.704131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2f187-42c4-41ae-9ae7-7b4e8418dea8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.743494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.744294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.744696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.744886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.757118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:01.765116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:01.766267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 290/585] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:01.772719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.787885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda744b4-09bc-4317-a903-26058cbd51d1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.799455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.825916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.826586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.826866 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.827142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.827856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:01.828159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.829585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.848020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 296/586] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:01.848894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.884073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:01.889159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d0cb-da17-4950-a700-e3bbe6ae06e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:01.934177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32530726-f3e6-445d-b862-876989f6692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:01.934856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32530726-f3e6-445d-b862-876989f6692e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:01.935112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32530726-f3e6-445d-b862-876989f6692e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:01.935395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32530726-f3e6-445d-b862-876989f6692e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:01.936179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32530726-f3e6-445d-b862-876989f6692e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:01.936528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32530726-f3e6-445d-b862-876989f6692e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:01.937973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32530726-f3e6-445d-b862-876989f6692e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:01.956060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 291/587] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:01.956060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32530726-f3e6-445d-b862-876989f6692e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:01.967501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8fdaf-329f-4941-9890-60ae0b8b1f4f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.012441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.013468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.013964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.014311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.015170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:02.015553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.017301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.037721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 297/588] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:02.038627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.068207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32530726-f3e6-445d-b862-876989f6692e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.080041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:02.127165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.128575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.129157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.129662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.131099 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:02.131671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.133692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.146862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 292/589] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:02.147612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.177973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a840f1-6cc6-473f-9f22-32da95201a66 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.179275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:02.225342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.226306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.226694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.227086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.235875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.238123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:02.256385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:02.257686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:02.258718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 298/590] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:02.259450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.295656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e2840-3f2a-4733-8488-7607a242671f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.358350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.359368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.359718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.360066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/535f23db24704422a4128ff09f1cc284/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.380631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=535f23db24704422a4128ff09f1cc284, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.382102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.393142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:02.398807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 293/591] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:02 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/535f23db24704422a4128ff09f1cc284/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:02.399739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.418148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad246d6-c2dd-48b5-a2c2-5a52aa5cab89 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.465440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.466490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.466783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.467068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/867e9b57fa7947e3afcbd974ed887969/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.486217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=867e9b57fa7947e3afcbd974ed887969, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.488666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.503829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:02.507557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 299/592] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:02 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/867e9b57fa7947e3afcbd974ed887969/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:02.508266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.576580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e850b6-d9eb-4867-af68-6c2cd7a655c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.661755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.663046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.663574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.664050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.677144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.679236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.698793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f60e61-6328-496f-b62e-50d8dc5ba904 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:02.704034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:02.705067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:02.705941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:02.705941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:02.705941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:02.705941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:02.708869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 294/593] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:02.716829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.743572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.744711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.744711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.745000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.745929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:02.746255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.747748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.760493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:02.779645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 300/594] 10.222.0.16 () {56 vars in 1137 bytes} [Thu May 26 13:31:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:02.782035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.866510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee4ed-f325-4534-aa9b-21cc960e716c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.897926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:02.943226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb62d3f-9c7e-4eee-a586-c32b90fc51a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:02.945726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.946425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.946719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.947004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:02.947749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:02.948065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:02.949640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:02.982959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 295/595] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:02.989766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:02.991461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:02.992282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:02.992605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:02.992888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.006294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.007793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:03.029152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:03.030310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:03.032030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:03.032030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:03.032030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:03.032030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:03.035398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 301/596] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:03.036670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.115070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:03.182375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99f6bf-5d2c-49c2-97e7-8cf06609ca65 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:03.233081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:03.234134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.234639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.235135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.236064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:03.236596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.239627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.263564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 302/597] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:03.270271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.285799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc562e-28f4-48e5-b8c1-b0d7e8a972df None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:03.334227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:03.335059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.335349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.335634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.357289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.358811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:03.389430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:03.390442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:03.391376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 296/598] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:03.396868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.414081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466e534-16bf-4e28-a0f9-c7593dc20fcd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:03.425815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:03.452479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:03.453330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.453623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.453909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.462409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.463897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:03.484536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:03.486577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:03.488229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 303/599] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:03.504839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:03.506308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.599113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2735c-e01f-4779-8e0f-97887ae48169 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:03.646624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060c40-150e-4eb2-9e0e-21da1f1a86f7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:03.650131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:03.650814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.651098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.651377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.652227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:03.652573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.654063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a024e5-38cf-46e6-b5b3-cf74db1706a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.675050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 297/600] 10.222.0.16 () {56 vars in 1139 bytes} [Thu May 26 13:31:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:03.676990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94593c-3554-43ab-a2b7-3db0100491d8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.677379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94593c-3554-43ab-a2b7-3db0100491d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.677769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94593c-3554-43ab-a2b7-3db0100491d8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.684590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 298/601] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:03.685902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94593c-3554-43ab-a2b7-3db0100491d8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.686576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:03.687262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.687548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.687879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.688717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:03.689033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.690768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa7caf-11e4-4a50-a08b-05b27076728a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.704678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 304/602] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:03.706585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.707090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.707378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.725909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:03.727238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:03.756553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:03.811011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 305/603] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:03.812155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.843906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:03.852435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94593c-3554-43ab-a2b7-3db0100491d8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:03.895788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:03.896756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:03.896942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:03.897225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:03.908796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:03.910360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:03.930622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:03.931531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:03.932469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:03.932469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:03.932469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:03.932469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:03.932469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 299/604] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:03.933108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:03.957411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b9267-5e1f-49d6-8b01-93d2643579c0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.000518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.002293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.002826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.003555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/4e8552f561ce4b4fa837f907a58816aa/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.023392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=4e8552f561ce4b4fa837f907a58816aa, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.025206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.042413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 306/605] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:03 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/4e8552f561ce4b4fa837f907a58816aa/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:04.043781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.047727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605f81d-b22e-4262-9382-524a403b63e2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.061107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:04.090041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.090876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.091624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.091624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.111650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:04.132086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:04.133356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:04.134431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:04.135233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 300/606] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:04.141902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.179851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:04.270123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b830c-6578-499b-80a4-0ad5cda50591 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.302793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215adaa5-587c-4f61-8a20-d40821d159c9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.307983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.308874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.309264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.309639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.310572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:04.310999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.312615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.326739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 307/607] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:04.332211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.339242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.339916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.340201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.340530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.341281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:04.341592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.342985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.352009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 301/608] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:04.357795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.441038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:04.442554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:04.470737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef37e14-927e-49a5-a07e-b9c4a9cdb58f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.484809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4bcc1-f36f-40ce-8d1b-e2e3d7f68fe0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.510377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.511182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.511464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.511742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.523137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.524702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.528125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.529301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.529592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.529874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.538153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.539659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:04.543733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:04.544674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:04.545564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:04.545564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:04.545564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:04.545564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:04.545564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 302/609] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:04.551658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:04.555662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:04.556764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:04.557678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 308/610] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:04.569485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.682791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bfdfb-5e72-49a5-8913-ab881c71987c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.716282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:04.728757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.729509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.729798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.730084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.730848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:04.731162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.733689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.760878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 303/611] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:04.764760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46b87-cb5c-41af-9b12-ec3269a2be5d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:04.765975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.814786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:04.825689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:04.826382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:04.826677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:04.826966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:04.827783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:04.828103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:04.829858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:04.858441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 309/612] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:04.867491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:04.978330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802126d-4b1d-47f0-bafa-ae7e21442f2f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:05.028617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760c0ce-acc5-4db7-aeb7-6cb5a10891f0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:05.067858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:05.068830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.069125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.069405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.069977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:05.070969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.071263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.071921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/6f44d10be1654d17990966ae64d43ebd/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.096255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:05.098254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=6f44d10be1654d17990966ae64d43ebd, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:05.100092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:05.101978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:05.124733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 310/613] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:04 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/6f44d10be1654d17990966ae64d43ebd/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9e14dce8-b76a-4d8a-8aa8-b2cae00141bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:05.132701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:05.133748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:05.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:05.137607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 304/614] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:05.145755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:05.214996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:05.306817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6239480-10b1-4e9f-a2ce-6e05d8318b10 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:05.311033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06467-a0bb-4069-a9bb-1d43b55c894e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.311372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06467-a0bb-4069-a9bb-1d43b55c894e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.311609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06467-a0bb-4069-a9bb-1d43b55c894e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.314831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 305/615] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:05.320021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a032-62cb-4d0d-a092-cbf3bc21baa5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.320510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a032-62cb-4d0d-a092-cbf3bc21baa5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.320829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a032-62cb-4d0d-a092-cbf3bc21baa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.351091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8263a032-62cb-4d0d-a092-cbf3bc21baa5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:05.352377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8263a032-62cb-4d0d-a092-cbf3bc21baa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:05.369154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:05.369862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.370148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.370423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.371286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:05.371599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:05.373115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:05.406658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 311/616] 10.222.0.16 () {56 vars in 1135 bytes} [Thu May 26 13:31:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:05.417409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:05.435683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:05.469549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 306/617] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:31:05.473795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc188f-80e7-49a1-b665-4be68502a480 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.474075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc188f-80e7-49a1-b665-4be68502a480 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.474358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc188f-80e7-49a1-b665-4be68502a480 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.475998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 307/618] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:05.481885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.482177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.482441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.501676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:05.503064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:05.557068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:05.562693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c0601-67a7-4d55-a535-50fdaf6999f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:05.580246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 308/619] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:31:05.581324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:05.614023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:05.615216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.615699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.616163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.629400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:05.630926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:05.650545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:05.651667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:05.652703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:05.652703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:05.652703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:05.652703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:05.652703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 312/620] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:05.653541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:05.746069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc8af3-1994-4b94-9158-b008be5f9152 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:05.758067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:05.798441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:05.799255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.799536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.800337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.805945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fdade9-2caf-40bb-a880-9220baa22fb9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:05.815855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:05.826330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:05.827343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:05.828185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:05.834780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 309/621] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:05.836009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:05.857257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:05.858780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:05.859281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:05.860338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:05.875336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:05.886865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:05.887874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:05.888761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:05.896400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 313/622] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:05.896400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:05.921830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:05.999043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc95e4-5618-4ae4-b8ed-b4c79c80fc12 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.046184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.047333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.047735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.048128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.049451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:06.050984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.052976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.058805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b46b01-5c24-4232-b50c-f0caf9aff748 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.077125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 310/623] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:06.078568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.110541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.112914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.113693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.114451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.115407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:06.116177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.118584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.119645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:06.132907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 314/624] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:06.134064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.217419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:06.217708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39553a9a-7254-484a-ab1e-69a0ffc75dde None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.253869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923e90-cc1d-4d5e-91e9-2530c3700c2e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.260673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.261445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.261808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.262160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.262998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:06.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.264869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.278939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 311/625] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:06.278939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.297572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.298674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.299097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.299519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.308005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.309728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:06.325393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:06.326378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:06.327262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:06.329836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 315/626] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:06.330978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.394249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:06.409607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd440b-7611-4ed0-85d7-fd3fa8659f1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.443046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a2e09-ae52-4f10-b26d-482caaa80256 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.447050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.447278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.447655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.448100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.461317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.463017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.481005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:06.483808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:06.484934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:06.485847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:06.485847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:06.485847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:06.485847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:06.490592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.491174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 312/627] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:06.491897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.492277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.492822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.493410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.512364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.512364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:06.533389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:06.535002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:06.536063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:06.536063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:06.536063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:06.536063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:06.536323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 316/628] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:06.537485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.607783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631faf62-e333-4303-9283-48d2f136b731 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.647384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.648175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.648594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.648983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.649892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:06.650313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.651892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.660896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50846d72-4c61-4d50-99df-492d9290c348 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.676940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:06.692041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 313/629] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:06.693851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.704129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.705130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.705565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.706566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.707456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:06.707937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.709781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.725860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 317/630] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:06.726933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.790818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:06.877115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a60e5-c63f-4354-b135-b9e45d5f1605 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.877944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be809f5f-6dc6-4e8d-b3e6-52b654cc36d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:06.927021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.928621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.929127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.930266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/e4e2c8a980714257b8df1d486d446eeb/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.943002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:06.944133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:06.944758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:06.945178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:06.946555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:06.947118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.949736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.951202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=e4e2c8a980714257b8df1d486d446eeb, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:06.953222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:06.971293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 318/631] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:06 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/e4e2c8a980714257b8df1d486d446eeb/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:06.972875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:06.985305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 314/632] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:06.988541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.099516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:07.109916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0361e2c-7067-499c-baf2-e4bc157f7020 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.141941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:07.163432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.165421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.165939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.166421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.178787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:07.180649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-18a8fce3-165e-4fd8-9dcd-e900709d1663 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:07.203022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:07.204083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:07.205077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:07.205635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86c5de-32e4-485a-aaad-3e6a975be551 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.208040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 319/633] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:07.210097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d095a-ee21-49b7-b708-e0cd2411fdaa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.210628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d095a-ee21-49b7-b708-e0cd2411fdaa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.211130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d095a-ee21-49b7-b708-e0cd2411fdaa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.213427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 320/634] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:07.220625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.220625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.220625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.238916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:07.240650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:07.286615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.287919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.288433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.288889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.303842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:07.306426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:07.324131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:07.325610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 315/635] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:07.326863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.342312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 321/636] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:07.343542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.448930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:07.469830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a776-55c1-4923-b385-4faa634760ac None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.470041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb81da-5566-44eb-a05e-653912127d7c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.495746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:07.511975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.512829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.513137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.513495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.514469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:07.514763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.514884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:07.515785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.516334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.516489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:07.516934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.518013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:07.519349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:07.521203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:07.538114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 316/637] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:31:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:07.539090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.544992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 322/638] 10.222.0.16 () {56 vars in 1137 bytes} [Thu May 26 13:31:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:07.551321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.654184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d64e49-2bd2-4984-b42e-e5991d29c5a8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.665978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873a16-0e3b-4c87-907e-464fdcfacfe3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.693088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.693902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.694187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.694469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.703765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66efa5a-c8d8-4b2e-b9f7-db5519e2f5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.704379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66efa5a-c8d8-4b2e-b9f7-db5519e2f5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.704672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66efa5a-c8d8-4b2e-b9f7-db5519e2f5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.704951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66efa5a-c8d8-4b2e-b9f7-db5519e2f5b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.706367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:07.716374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:07.717559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:07.718474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 317/639] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:07.719174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.721997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:07.763340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:07.794896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66efa5a-c8d8-4b2e-b9f7-db5519e2f5b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:07.796545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66efa5a-c8d8-4b2e-b9f7-db5519e2f5b9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:07.872258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9935d9-9840-4128-985d-ff8cd20eb40c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:07.924980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 323/640] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:31:07.924980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93626a7-a15c-4e87-a779-293320fb72e0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.924980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93626a7-a15c-4e87-a779-293320fb72e0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.924980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93626a7-a15c-4e87-a779-293320fb72e0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.924980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 324/641] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:07.925946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93626a7-a15c-4e87-a779-293320fb72e0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:07.926343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfbe17-d094-4b62-80ac-492024845ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:07.927366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfbe17-d094-4b62-80ac-492024845ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:07.927663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfbe17-d094-4b62-80ac-492024845ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:07.927944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfbe17-d094-4b62-80ac-492024845ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:07.997055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.016584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfbe17-d094-4b62-80ac-492024845ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.018097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfbe17-d094-4b62-80ac-492024845ba2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.077941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.082943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93626a7-a15c-4e87-a779-293320fb72e0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.126599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.127270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.127557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.128000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.128910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:08.129323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.130753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.141683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 325/642] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:08.142345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.171858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 318/643] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:31:08.173971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.174522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.175093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.192568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:08.193805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:08.273945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 319/644] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:08.274771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.292886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19fcaa-26b2-403b-9188-5ab982def42d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.296740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.339178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.339994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.340275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.340578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.355469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.356990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.374876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:08.375469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:08.377114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:08.379404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:08.379404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:08.379404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:08.379404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:08.382894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 326/645] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:08.383873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.436031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5f6c2-bae0-43c0-82bc-3dc29b261a31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.477758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.478882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.479328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.479759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.493351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.495214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.509008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673eb50-7e25-466e-acd2-e5503f6bfe82 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:08.523151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:08.524363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:08.525639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:08.525639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:08.525639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:08.525639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:08.527861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 320/646] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:08.529342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.558932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.559856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.560208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.560597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.574603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.576068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:08.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:08.598757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:08.599920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:08.599920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:08.599920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:08.599920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:08.599920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 327/647] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:08.599920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.614838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.664257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f30d7-273b-40e5-b2e7-514ddf3c1ba7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.692527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.709909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.711562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.712079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.712578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.730034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:08.742419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:08.743635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:08.744656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 321/648] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:08.745637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.757758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2b7a-e264-4feb-8c5a-bcd7e67a4ed7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.812891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.813519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.813784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.814241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.815022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:08.815951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.817774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.839433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 328/649] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:08.839433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.890475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41d5b9-9c27-4648-bf62-832b7e2eaf16 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:08.931924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:08.939440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:08.940739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:08.941097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:08.941450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:08.942479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:08.942805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:08.945369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:08.964345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 322/650] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:08.964345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:08.978827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed15bef-6e11-4757-bfb5-5e0e897846a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.015948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.017122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.017609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.018064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.018971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:09.019460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.021134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.024243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.058847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 329/651] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:09.058847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.109407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b098-d115-49ce-b985-85e4aa564dc5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.158738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.159979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.160264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.160585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.161343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:09.161655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.163090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.177348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 323/652] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:09.177348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.215022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ab24c-0650-43b0-bd93-c010253c1db7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.245426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.255391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.256200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.256511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.256791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.263766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.265269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:09.279606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:09.280832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:09.281785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:09.286325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 330/653] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:09.287005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.295516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33c94c-ea43-4596-aa48-f943325bfcd2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.297247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.347912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.348934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.349404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.349866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.362489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.364866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:09.389773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:09.390774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:09.392267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 324/654] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:09.394453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.432524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1886d-e2f1-47e0-aa06-b436208b104e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.473577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.474675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.475154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.475427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.483318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.484868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:09.500328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:09.501467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 331/655] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:09.502375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.529319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4651e83-0760-4065-ba22-ab42d0e399d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.560953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.567472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.568280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.568603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.568870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.579206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.581439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:09.602106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:09.603065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:09.604660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:09.604660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:09.604660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:09.604660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:09.604660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 325/656] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:09.604660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.621166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.625069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31445af4-2695-478f-b350-f80282bac9a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.663354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.664022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.664330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.664616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.665405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:09.665712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.667160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.679847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 332/657] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:09.680688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.708513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86693f55-286c-4ab6-b756-78d1bd1d030f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.746251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.746925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.747216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.747497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.748361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:09.748692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.750142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.774503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 326/658] 10.222.0.16 () {56 vars in 1137 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:09.774503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.799016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b6fa6-a716-42ee-9fa9-aaf9cbb1c182 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.809982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.835700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.836396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.836694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.836968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.837753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:09.838064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.839521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce9816-2bce-4964-8028-b7c88807b0c8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.852275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 333/659] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:09.853606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86a08d-91fb-43c3-8331-41e531c848c8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.853897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86a08d-91fb-43c3-8331-41e531c848c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.854277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86a08d-91fb-43c3-8331-41e531c848c8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.856117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 334/660] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:09.856262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86a08d-91fb-43c3-8331-41e531c848c8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.856772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:09.889118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625f6be-01a7-46b4-8f50-6cf1372d5934 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:09.924257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:09.925597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:09.925883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:09.926171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:09.926895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:09.927208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:09.928657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:09.940048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 327/661] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:09.940906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:09.963081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86a08d-91fb-43c3-8331-41e531c848c8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.002013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.002901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.003352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.003799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.005278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=535f23db24704422a4128ff09f1cc284) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:10.005761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.008332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.015269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 335/662] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/role_assignments?user.id=535f23db24704422a4128ff09f1cc284&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:10.016132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.049218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f25f1-dada-4a37-87bf-bb8ec58eb548 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.063730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.085564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.091356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.092014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.092295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.092604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.155943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79968990-496f-45f5-9b5d-9375f10a580b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.182627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.184141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.201564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.202329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.202610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.202887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.203846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=867e9b57fa7947e3afcbd974ed887969) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:10.204280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.206357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3def4-6dc7-4ada-a249-ead7d5982ad7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.220739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 336/663] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/role_assignments?user.id=867e9b57fa7947e3afcbd974ed887969&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:10.222789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.223081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.223362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.248006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:10.249313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:10.345581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 328/664] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:31:10.347142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.353557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 337/665] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:10.355446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.392051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.408919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.478833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e165fe-a8bb-43ba-98a8-6cfa64bfb423 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.491482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d240e-685a-42ce-9009-d51f70a15540 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.529886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.531059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.531640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.532252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.537824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.538933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.539401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.539862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.540935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.542528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.552624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.554436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:10.559046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:10.560012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 13:31:10.561015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:10.564689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 329/666] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:10.565564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:10.577872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:10.578832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:10.579724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:10.579724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:10.579724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:10.579724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:10.584078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 338/667] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:10.585008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.666610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.676332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eba74-7eab-4b92-800f-e8cde41acaff None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.678088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.704489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674e83-e8a4-40ea-907b-39ae77556b3c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.714136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.715128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.715515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.715892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.727802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:10.735819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:10.737016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:10.738233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:10.742536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 330/668] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:10.743466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.749601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.750608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.751099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.751570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.753387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:10.754185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.756640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.781906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 339/669] 10.222.0.16 () {56 vars in 1139 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:10.784626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.862704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a29eca-16eb-4993-8839-d2eecd9950cf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.905618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.906312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.906596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.906873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.907597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:10.907906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.912335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.923282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 331/670] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:10.924877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:10.938041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65eaa0-292c-4446-ac3e-64af372ded5d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:10.961469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.975696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:10.976398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:10.976684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:10.976961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:10.977706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:10.978010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:10.979401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:10.982228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:10.995167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 340/671] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:10.996474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.054883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46393e-0810-4671-a512-3455c10e6e0c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.100298 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.101318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.101606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.101885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.119467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab641a-0c1b-4ea9-af0f-a63214123e35 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.158344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.159675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.160143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.160612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.168680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.170407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:11.186480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:11.187628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:11.188662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:11.189098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.190552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.191372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 341/672] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:11.192523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.209539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:11.264214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:11.322452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6340f9-4e96-4b9b-8767-026f7e7e4f60 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.326096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 332/673] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:31:11.327879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.370239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.371305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.371590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.371871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.388406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.390167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:11.413510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:11.414640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:11.415775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:11.415775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:11.415775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:11.415775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:11.415775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 342/674] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:11.416119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.464228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625814d-f740-4d8e-8e12-1d17b702adc4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.483360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:11.517223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.518414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.519514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.520008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.521507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:11.522312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.524743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.548559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 333/675] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:11.550421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.560818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bc495-abe7-494d-99cc-5ae6f7755080 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.577119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:11.604572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.605382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.605672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.605953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.617777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.619242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:11.638116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:11.639089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:11.640535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:11.640535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:11.640535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:11.640535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:11.640535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 343/676] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:11.644551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.676001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca20183-46e0-4445-a95f-a901aaa8dec7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.713744 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.714839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.715330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.715813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.716784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:11.717381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.719029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.733693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 334/677] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:11.735043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.753533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb372-bf06-48fa-8e5c-0ae877e8c5ec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.770864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:11.792989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.793692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.794247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.794358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.795198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=4e8552f561ce4b4fa837f907a58816aa) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:11.795519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.797007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.812347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 344/678] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/role_assignments?user.id=4e8552f561ce4b4fa837f907a58816aa&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:11.812347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.841898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:11.887240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bdbd7-f4ca-4e12-9be4-2ba3b7724e39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.942250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07742-3651-402f-afd3-55666643fc80 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:11.957188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.958684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.959229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.959720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.960714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:11.961301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.962928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:11.984337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:11.984337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:11.984337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:11.984337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:11.985068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 335/679] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:11.989626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:11.997000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:11.998484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:12.027305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:12.028453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:12.030173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 345/680] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:12.034872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:12.152895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:12.169001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf9223-6489-4b02-8a7d-11d03c2ba997 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:12.219975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:12.222606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.223085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.223860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.235593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:12.238624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:12.256692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:12.261906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:12.262815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:12.263779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:12.264217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ff3c7-c282-4f30-8915-7a5b08177d58 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:12.267528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 336/681] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:12.273322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:12.314772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:12.316022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.316338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.316618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.317483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:12.317795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:12.319261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:12.348569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 346/682] 10.222.0.16 () {56 vars in 1135 bytes} [Thu May 26 13:31:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:12.354885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:12.435061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9909be-7370-4b83-b00d-b7ce1aea0593 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:12.483717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a17518-3f47-4443-baf0-c17b369026a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:12.486262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:12.488020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.489466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.489971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.490986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:12.491532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:12.493266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:12.511640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 337/683] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:12.521657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:12.539727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:12.541290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb08b2-239a-493e-9e45-fe4cd3399a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:12.542067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb08b2-239a-493e-9e45-fe4cd3399a58 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.542556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb08b2-239a-493e-9e45-fe4cd3399a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.542769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb08b2-239a-493e-9e45-fe4cd3399a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.569713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:12.632163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb08b2-239a-493e-9e45-fe4cd3399a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:12.633816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb08b2-239a-493e-9e45-fe4cd3399a58 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:12.664578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57209af7-54bd-4763-8366-38b66d7eaa04 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:12.714624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:12.715641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.716118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.716612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.717648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=6f44d10be1654d17990966ae64d43ebd) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:12.718183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:12.723187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afd8fc-bb58-4ca0-9a8e-8aebb3bffb6c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:12.732595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 338/684] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:12 2022] GET /identity/v3/role_assignments?user.id=6f44d10be1654d17990966ae64d43ebd&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:12.734278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2b28e-9848-4f44-b325-71a17ec48d48 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.734739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2b28e-9848-4f44-b325-71a17ec48d48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.735216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2b28e-9848-4f44-b325-71a17ec48d48 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.737142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 339/685] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:12.744870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed436f9-926f-498d-b06d-ea229a942083 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.744870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed436f9-926f-498d-b06d-ea229a942083 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.744870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed436f9-926f-498d-b06d-ea229a942083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.762893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bed436f9-926f-498d-b06d-ea229a942083 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:12.764117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bed436f9-926f-498d-b06d-ea229a942083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:12.783387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 347/686] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 13:31:12.786021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17e92-7bb3-459a-b0d5-a6fec0b17294 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.786382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17e92-7bb3-459a-b0d5-a6fec0b17294 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.789678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17e92-7bb3-459a-b0d5-a6fec0b17294 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.789678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 348/687] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:12.793009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff17e92-7bb3-459a-b0d5-a6fec0b17294 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:12.854625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:12.864394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 340/688] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:31:12.866147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.867703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.868296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.888997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:12.894559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:12.895750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:12.934937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff17e92-7bb3-459a-b0d5-a6fec0b17294 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:12.975510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:12.976799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:12.977295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:12.977800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:12.980663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 341/689] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:12.981730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:12.991894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:12.993721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:13.013846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:13.014802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:13.015698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:13.015698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:13.015698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:13.015698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:13.018071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 349/690] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:13.018547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.104591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d81a3-cc1c-408b-9c13-036d4d12b81b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.138382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:13.146578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f6189-072f-49f4-acd2-bd2da0f26f2d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.157992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.159310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.159999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.160951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.165073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:13.178296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:13.188973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:13.190790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:13.191993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:13.198975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.198975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.198975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.198975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.199418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.199418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.199635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 342/691] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:13.200733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:13.216042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:13.217293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:13.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:13.220096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 350/692] 10.222.0.16 () {56 vars in 1129 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:13.226117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.335917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ccf73-a479-4dde-92a5-e2da938a7a04 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.362960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52e196-f8f0-4fdb-b640-a530578d5b78 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.386569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.388197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.388759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.389539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.390505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:13.391230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.393597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.409444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 343/693] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:13.410646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.412501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.413774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.414311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.414827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.416247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:13.421307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.423707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.442035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 351/694] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:13.442035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.469659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:13.503443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:13.561840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1312-2dcd-4974-ace9-e10127bf60b0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.565812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb7a79-a40e-4880-8c38-8550643dcfd6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.607619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.608212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.608527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.608792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.608911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.609523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:13.609831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.610364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.610851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.611225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.611556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.620562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.621353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 344/695] 10.222.0.16 () {56 vars in 1124 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:13.622379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.623011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:13.642405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:13.643718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:13.644658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:13.648651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 352/696] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:13.649799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.740671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e322cd-93a9-4a6f-a7f5-7b651d76e736 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.760240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:13.764040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:13.781242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.782104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.782391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.782706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.784904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fab55-d556-4491-b216-435fe66a19e1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:13.794771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.796350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:13.818888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:13.819919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:13.820868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:13.820868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:13.820868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:13.820868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:13.824691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 345/697] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:13.825651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.828726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:13.829608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:13.831372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:13.831665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:13.832473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:13.832794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:13.834458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:13.842724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 353/698] 10.222.0.16 () {56 vars in 1124 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:13.843576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:13.979196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688cf01-8fc4-4727-a05d-8d270fa4e3e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:14.023320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:14.024003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.024291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.024761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.025502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:14.025813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:14.027477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:14.043534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea3090-af95-4f3b-816f-9d323ab14901 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:14.058772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 354/699] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:14.064497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:14.082488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:14.094901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:14.097414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.097973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.098563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.099677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:14.100231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:14.102270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:14.117687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 346/700] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:14.123190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:14.194178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:14.220212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40d7eb-fb57-486d-8b71-58a77daec594 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:14.250020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf122b-a167-469c-a8ad-8dd146a807e9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:14.257492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:14.258301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.258585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.258861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.266742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:14.268791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:14.284330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:14.285315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:14.286267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:14.286876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 355/701] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:31:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:14.287440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:14.288069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.288743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.289098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.290000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=e4e2c8a980714257b8df1d486d446eeb) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:14.290324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:14.291771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:14.293519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:14.299751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 347/702] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:14 2022] GET /identity/v3/role_assignments?user.id=e4e2c8a980714257b8df1d486d446eeb&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:14.368509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:14.405183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877e213-e469-43d0-b05c-67e9a8204be1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:14.444652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:14.445253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.445539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.445815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.446661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:14.446969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:14.448450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7215-70d6-44fb-b05c-a9e958524335 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:14.470341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 356/703] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:31:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:14.476186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:14.621096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:14.627728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14817-81ef-4c7f-a48c-ae10e965873c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:14.670412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:14.671127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.671412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.671742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.731885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4133c-46b5-4dda-8168-b55fc82d68c6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.732518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4133c-46b5-4dda-8168-b55fc82d68c6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.733032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4133c-46b5-4dda-8168-b55fc82d68c6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.736402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 357/704] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:14.741178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:14.741454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:14.741735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:14.759144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:14.760652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:14.780474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:14.781750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:14.870658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:14.879064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 348/705] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:31:14.884568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:14.893897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 358/706] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:14.926624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.028805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:15.030933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85816-e750-4921-8c32-ef86f9964b32 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:15.071685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:15.072519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.072806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.073088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.086673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:15.088240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:15.110724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:15.111860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:15.113327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:15.113327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:15.113327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:15.113327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:15.113327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 349/707] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:15.118424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.166117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c89d03-747f-4b9a-ae79-e31b5e124b23 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:15.227892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:15.244467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:15.245476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.245769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.246124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.260738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:15.262441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:15.294171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:15.295118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:15.296033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:15.296033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:15.296033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:15.296033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:15.305577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 359/708] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:15.317888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.331299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6f26d-d686-4d07-848f-938adddb505b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:15.379538 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:15.381544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.382732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.383375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.385465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:15.386202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:15.388478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:15.417555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 350/709] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:15.423785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.547511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af3e41-3862-4db6-b6b2-ce0634bf92f2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:15.559537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:15.576018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa47317-efbd-44e2-96fd-5b94a12aba1a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:15.586919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:15.612457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:15.613348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.613571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.613913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.614597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:15.614907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:15.616362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920cc5b-ebf5-4fc4-89e5-556545b4f78e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:15.636965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 360/710] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:15.638654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1518f21-a4d3-485d-a67b-b4495c2497e0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.638942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1518f21-a4d3-485d-a67b-b4495c2497e0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.639226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1518f21-a4d3-485d-a67b-b4495c2497e0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.645180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:15.645180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.645180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.645180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.645582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 361/711] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:15.645657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1518f21-a4d3-485d-a67b-b4495c2497e0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.653813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:15.655778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-909a7e90-8313-4849-ade5-b6aabbb8b8d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:15.672169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:15.673149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:15.674047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:15.682418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 351/712] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:15.684201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.684737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.685227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.703578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:15.705183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:15.787108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 352/713] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:15.788410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.789328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1518f21-a4d3-485d-a67b-b4495c2497e0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:15.844790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:15.845395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:15.845681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:15.846466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:15.847203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:15.848701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:15.871926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 362/714] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:31:15 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:15.873527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:15.908007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:15.940484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:15.954132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597e0e5-8b52-4418-9eb6-8f0c37bea6c9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:16.021975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:16.022956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.023397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.023827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.024847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:16.025319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:16.026972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:16.041506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 353/715] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:16.048148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:16.100880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cb874-497f-46a4-9fd2-f44078ca640e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:16.173455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:16.176024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.176739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.179672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.179672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:16.179889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26fe9-e969-44b1-ad7d-686f41f3a8bc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:16.179971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:16.198515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:16.199611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:16.200545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:16.207896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 363/716] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:15 2022] GET /identity/v3/services/network => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:16.211259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:16.235950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:16.237253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.237594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.237839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.238652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=535f23db24704422a4128ff09f1cc284) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:16.239037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:16.240583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:16.248172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 354/717] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:16 2022] GET /identity/v3/role_assignments?user.id=535f23db24704422a4128ff09f1cc284&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:16.337927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:16.345349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2695ca-604b-4154-98f5-6c488dfc6e7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:16.389414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:16.390393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.390838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.391271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.392186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:16.392699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:16.394422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ae5f-6f54-4b8e-8ad3-20a9e790aeb2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:16.404462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 364/718] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:16.407866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:16.467969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:16.587395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187acb9-3ed4-4b75-8c03-0da40b713fef None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:16.642920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:16.643592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.643880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.644159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.644906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:16.645217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:16.646727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be0ce9-2831-4d3f-ba78-a78120d9c720 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:16.657723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 355/719] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:16.675133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2f26-5c4a-45e9-b7e4-60b4a4b4c812 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.676184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2f26-5c4a-45e9-b7e4-60b4a4b4c812 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.676743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2f26-5c4a-45e9-b7e4-60b4a4b4c812 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.680105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 365/720] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:16.685126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d5893-15b7-414d-b7c1-8eb2ffb3deea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:16.685417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d5893-15b7-414d-b7c1-8eb2ffb3deea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:16.685708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d5893-15b7-414d-b7c1-8eb2ffb3deea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:16.707689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5d3d5893-15b7-414d-b7c1-8eb2ffb3deea None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:16.709181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5d3d5893-15b7-414d-b7c1-8eb2ffb3deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:16.804288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 356/721] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:16.848066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2f26-5c4a-45e9-b7e4-60b4a4b4c812 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:17.009268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2f26-5c4a-45e9-b7e4-60b4a4b4c812 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:17.053827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5040c4-3080-4ff5-87f8-db90bddd78e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:17.054507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5040c4-3080-4ff5-87f8-db90bddd78e7 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:17.054790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5040c4-3080-4ff5-87f8-db90bddd78e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:17.055068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5040c4-3080-4ff5-87f8-db90bddd78e7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:17.055802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5040c4-3080-4ff5-87f8-db90bddd78e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:17.057231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5040c4-3080-4ff5-87f8-db90bddd78e7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:17.076601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 366/722] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:16 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:17.408993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583cbe2-2488-41c2-9b0b-dc1d11a2cda1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:17.409408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583cbe2-2488-41c2-9b0b-dc1d11a2cda1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:17.409535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583cbe2-2488-41c2-9b0b-dc1d11a2cda1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:17.411270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 357/723] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:17.417547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3137-99cd-49b3-906c-821294166d9e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:17.418190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3137-99cd-49b3-906c-821294166d9e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:17.418678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3137-99cd-49b3-906c-821294166d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:17.442119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-497b3137-99cd-49b3-906c-821294166d9e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:17.443684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-497b3137-99cd-49b3-906c-821294166d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:17.449522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:17.565105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 367/724] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:17.589808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583cbe2-2488-41c2-9b0b-dc1d11a2cda1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:17.620768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:17.733021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583cbe2-2488-41c2-9b0b-dc1d11a2cda1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:17.781099 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:17.782263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:17.782753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:17.783232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:17.800495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:17.812906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:17.813904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:17.815699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:17.819460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 358/725] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:17.825440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3137-99cd-49b3-906c-821294166d9e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:18.029111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3137-99cd-49b3-906c-821294166d9e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:18.079643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:18.080588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:18.080787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:18.080998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:18.081935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:18.082247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:18.084082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:18.089796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:18.099540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 368/726] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:18.104846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:18.247016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:18.284290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b64d-4d6a-43b6-b31b-f3521e4bdeac None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:18.329989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:18.331572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:18.331911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:18.332191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:18.348518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:18.349999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:18.368776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:18.369790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:18.371278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:18.371278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:18.371278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:18.371278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:18.371278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 359/727] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:18.375633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:18.508009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46eb6b-3aae-4b4c-a7e5-19fb19e1263b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:18.544619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:18.545208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:18.545490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:18.545765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:18.546489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:18.546793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:18.548168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:18.561195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 369/728] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:18.569613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:18.682986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6982d-2efa-4cd7-a904-0c4f812d5673 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:18.710437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:18.725911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:18.726720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:18.727005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:18.727285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:18.736756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:18.738282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:18.754106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:18.755272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:18.756474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:18.758863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 360/729] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:18.765675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:18.852752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:18.914836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e8940-efd1-4e09-85dd-2c254fabf26b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:18.969945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:18.970637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:18.971468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:18.971750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:18.972625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:18.972939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:18.974383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56911499-defb-4c67-8a42-90bd0ddc96d0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:19.003309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 370/730] 10.222.0.16 () {56 vars in 1135 bytes} [Thu May 26 13:31:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:19.009114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:19.117097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b2d6b-12d8-4dd1-bb37-aeaffbddfb36 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.117304 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b2d6b-12d8-4dd1-bb37-aeaffbddfb36 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.117837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b2d6b-12d8-4dd1-bb37-aeaffbddfb36 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.120968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 371/731] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:19.125674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d747ae6-3c73-4b0a-8a90-4b00bc5d0ee7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.126229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d747ae6-3c73-4b0a-8a90-4b00bc5d0ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.126834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d747ae6-3c73-4b0a-8a90-4b00bc5d0ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.139561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11625e-4194-43ff-8585-ddc68d4891a6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:19.159883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5d747ae6-3c73-4b0a-8a90-4b00bc5d0ee7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:19.161288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5d747ae6-3c73-4b0a-8a90-4b00bc5d0ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:19.186880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:19.187789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.188220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.188660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.259815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 372/732] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:19.261647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9590-5b5f-43f4-9061-7db9f3bb0557 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.262138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9590-5b5f-43f4-9061-7db9f3bb0557 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.262788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b9590-5b5f-43f4-9061-7db9f3bb0557 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.264531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 373/733] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:19.271488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.272701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:19.272832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.273187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.291620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:19.293364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:19.295460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:19.297875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:19.352496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:19.393364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 374/734] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 13:31:19.394803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:19.437062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 361/735] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 13:31:19.438834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:19.546340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b9b69-7614-4d06-a85c-6afe32ae23ea None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:19.551056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:19.567049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e408b3-dcea-4532-b911-41deaed6f8f1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:19.607723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:19.609011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.609580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.610023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.622329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:19.630941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:19.631963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:19.633224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 362/736] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:19.633922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:19.716883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f391db2-294a-4ac2-b7d6-1f6e23fbb057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:19.717694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391db2-294a-4ac2-b7d6-1f6e23fbb057 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.717976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391db2-294a-4ac2-b7d6-1f6e23fbb057 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.718288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391db2-294a-4ac2-b7d6-1f6e23fbb057 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.729161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f391db2-294a-4ac2-b7d6-1f6e23fbb057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4f391db2-294a-4ac2-b7d6-1f6e23fbb057 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:19.737166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:19.738168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:19.739116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 375/737] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:19.739825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734ef63-15c2-4138-9263-0f6dbfa12718 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.739825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734ef63-15c2-4138-9263-0f6dbfa12718 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.740613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734ef63-15c2-4138-9263-0f6dbfa12718 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.741518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 376/738] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:19.745475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734ef63-15c2-4138-9263-0f6dbfa12718 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:19.751807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad62a22-c6ca-427f-8ad1-88c9c5607f7c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:19.789583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:19.790382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.790492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:19.790684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.790948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.802906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:19.804411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:19.813503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:19.824395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:19.825612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:19.826515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:19.826515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:19.826515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:19.826515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:19.826515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 363/739] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:19.826515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:19.870388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734ef63-15c2-4138-9263-0f6dbfa12718 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:19.925854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:19.926551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.926841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.927127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.928012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:19.928398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:19.929923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd2d14-2175-4bdb-83b0-340d1d70a3f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:19.944443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 377/740] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:19.944846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33125f-df44-4cef-a8c3-a06d2f623a6b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:19.945982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.946315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.946671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.965643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:19.967032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:19.982423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:19.983096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:19.983380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:19.983654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:19.984422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:19.984733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:19.986226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:19.996217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 364/741] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:19.997071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.052185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 378/742] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 13:31:20.052185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.074338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.086627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.117980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a312d25-e582-4284-b7bb-ddc2845c4b72 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.152994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.153929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.154211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.154488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.155207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:20.155520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.156955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.161864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d344cc-1ae4-41c3-bb7a-8609055b22e2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 365/743] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:20.169882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.210246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.211044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.211327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.211605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.222279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.223670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:20.241695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:20.242920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:20.244159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:20.244159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:20.244159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:20.244159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:20.244159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 379/744] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:20.244159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.282107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088a1d4-047c-4a7d-b218-a5ae3e320bb1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.306517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.318077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.318881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.319163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.319446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.328250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.329953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.331484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.348197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8233d-9237-4b7b-a3f4-43587cf39a6b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9195b5cd-bf9a-4762-baf3-4bc98f5f39da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:20.349743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:20.350694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:20.351573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:20.351573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:20.351573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:20.351573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:20.351573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 366/745] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:20.352423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa037f-848c-441f-9151-c8580940deba None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.353135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa037f-848c-441f-9151-c8580940deba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.353135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa037f-848c-441f-9151-c8580940deba None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.354319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 367/746] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:20.355037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa037f-848c-441f-9151-c8580940deba None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.390709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.391397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.391687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.391970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.392809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.394377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.407220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 380/747] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:20 2022] POST /identity/v3/services => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:20.408025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.480655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa037f-848c-441f-9151-c8580940deba None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.527570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.528427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.528715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.529141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.539057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.540589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.546049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478cacc-0c45-4d20-b2b6-3f10393885c8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:20.557228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:20.558303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:20.559217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 368/748] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:20.559845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.598134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.598826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.599114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.599399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.600129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:20.600481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.601894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7f53e-ec1e-44b2-ac3e-3a39dcca977a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.604046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.613886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.622371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 381/749] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:20.623869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.624803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.624803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.648400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:20.649622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:20.696404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fdfee-5794-4ca8-b46f-04ba65fe36f6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.741640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.742234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.742516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.742797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.743578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:20.743891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.745421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.750969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 382/750] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:20.750969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.774628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 369/751] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:20.775616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:20.887648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.900097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660048e-6989-459a-88fe-4217ca043c40 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.901486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc39004-3c1c-4dd5-b868-a7df9b7952ea None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:20.957847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.959027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.959460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.960127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.960503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:20.965567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:20.966615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:20.967077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:20.967537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:20.968940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:20.969702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.973779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.974180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:20.976644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:20.998836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 383/752] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:20.998836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.111640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe77bf6-4742-41de-b8c7-48e1b48b299c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:21.118070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:21.119026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:21.120000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 370/753] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:21.120685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.152657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.153474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.153762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.154037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.154687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.156070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:21.168785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:21.169748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:21.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 384/754] 10.222.0.16 () {56 vars in 1129 bytes} [Thu May 26 13:31:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:21.171476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.231297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:21.231490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803a5aa-f4cd-4a5f-9238-bdc2a6c4d609 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.269188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.270049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.270333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.270611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.278618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.280072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.281128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cbfb0-4165-4927-8fa2-dbf2c4c320c1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:21.296006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:21.297357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:21.298314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 371/755] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:21.298948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.311023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:21.326447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.327115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.327403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.327685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.328551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=e4e2c8a980714257b8df1d486d446eeb) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:21.328870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.330621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.344341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 385/756] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/role_assignments?user.id=e4e2c8a980714257b8df1d486d446eeb&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:21.344341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.416459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45fa21-800e-48fc-95d2-2e195acbe328 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.458687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.459384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.459665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.459944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.460832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:21.461258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.463592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.483729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a03e4-6dd1-46bd-9c0c-dab59d5ccdcb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.488414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 372/757] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:21.489430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.505123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:21.526904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.527929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.528444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.528940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.530079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:21.530701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.533109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.544443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 386/758] 10.222.0.16 () {56 vars in 1124 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:21.545779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.571093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:21.612368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a344d-909a-4583-90c7-ac0f7262a138 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.652287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.653035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.653322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.653633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.654667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:21.655009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.656652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.682145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6ea10-647f-4dde-bb6f-57e8dc6cf5dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.682731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 373/759] 10.222.0.16 () {56 vars in 1149 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:21.683538 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.720397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.722171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.722463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.722743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.735315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.737459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:21.755759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:21.758079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:21.759910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:21.759910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:21.759910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:21.759910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:21.759910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 387/760] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:21.759910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.795338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700bc1c-efdd-4d0b-8839-336becaac342 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.796276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:21.825555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:21.836088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.836855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.837142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.837417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.838175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:21.838487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.839973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:21.850743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 374/761] 10.222.0.16 () {56 vars in 1124 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:21.851543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:21.868485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd57e-a4e5-4267-8579-13b0272d8e78 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.910214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:21.910968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:21.911144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:21.916328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:21.978518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b27cb-d029-4551-bb62-ae14fc82d1d6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:21.995179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:21.996686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.014532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.015207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.015489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.015766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.016516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:22.016824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.018236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.032505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.035085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 375/762] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:22.036036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.102616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.125897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 388/763] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:31:22.126931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.152706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b65d1-c8bb-4202-85d6-1af7f17b5a8b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.191435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.192108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.192459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.192741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.193534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] RBAC: Adding query filter params (interface=public, service_id=50d6e67f4efa40ba8e8dfc587cef8461, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:22.193849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.195246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.203590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 376/764] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/endpoints?service_id=50d6e67f4efa40ba8e8dfc587cef8461&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:22.204533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.242388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7533d-0000-4f6b-a358-338dab2c7fa8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.281986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.283022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.284205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.284754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.287231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.296413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.297893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:22.313516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:22.314979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 389/765] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:22.316077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.320009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb93f1-42f7-4ae2-8d3c-2887e943bca6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.335551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.357325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.358147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.358505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.358854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.370578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.372151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:22.389927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:22.391835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:22.394466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:22.394466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:22.394466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:22.394466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:22.394466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 377/766] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:22.394466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.431710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b532c-70cd-4a57-a20c-f512611c5102 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.470133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.471007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.471509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.471791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.472506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.473988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.480088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 390/767] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:22.481055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.519646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04e6c9-091d-4165-8d52-722281aa9e3c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.617261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.617956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.618246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.618533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.619436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:22.619772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.621285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.622210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.642062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 378/768] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:22.648120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.699870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.709555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9703e-95f4-467a-a8cd-cab123e89c2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.751934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.752794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.753020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.753269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.754049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:22.754311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.755824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a2f6-13a7-483c-94b1-b280342cadb8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.769807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 391/769] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:22.771287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4a5ba-b290-4ea4-8ce8-1df9a713c82e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.771582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4a5ba-b290-4ea4-8ce8-1df9a713c82e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.771861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4a5ba-b290-4ea4-8ce8-1df9a713c82e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.773390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 392/770] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:22.776220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4a5ba-b290-4ea4-8ce8-1df9a713c82e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:22.795578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7d180-61eb-4a4d-b16c-e8279726e89f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.832351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed26ee-5147-45a7-a581-43b650bf81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.833540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed26ee-5147-45a7-a581-43b650bf81d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.833898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed26ee-5147-45a7-a581-43b650bf81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.834249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed26ee-5147-45a7-a581-43b650bf81d5 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/535f23db24704422a4128ff09f1cc284/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.836336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed26ee-5147-45a7-a581-43b650bf81d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=535f23db24704422a4128ff09f1cc284, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.837964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed26ee-5147-45a7-a581-43b650bf81d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.853494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 379/771] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:22 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/535f23db24704422a4128ff09f1cc284/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:22.855200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.855649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.856714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.876819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:22.878539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:22.919725 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4a5ba-b290-4ea4-8ce8-1df9a713c82e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:22.960264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 380/772] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:22.969143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:22.981748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:22.982879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:22.983334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:22.983762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:22.992902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:22.994983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:22.995206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:23.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:23.020774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:23.021970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:23.022540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:23.024366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 393/773] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:23.034006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.144418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89bbb-4c3f-42b4-96f3-30a2d9437c60 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:23.186000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:23.188124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.188556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.189036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.190156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:23.191562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:23.205712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802962c1-18e3-439f-a79b-f147ab76d2a2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:23.210842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:23.211938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:23.213578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 381/774] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:22 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:23.218025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.249123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:23.250123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.250600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.251062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.252862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:23.253455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:23.255559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:23.270024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 394/775] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:23.278101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.340614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:23.375472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:23.412609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84388c0c-3db2-4c73-8f0b-9b3464b2f200 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:23.446567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9ca6d-8e19-4c35-a109-aff2c751a59e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:23.454840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:23.455702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.456154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.456530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.457324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:23.457709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:23.459440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:23.465387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 382/776] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:23.471944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.522677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:23.523345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.523632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.523914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.524772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=53d524fa52bd42cea09081345ff8d5e6) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:23.525086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:23.526530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:23.534256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 395/777] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:23 2022] GET /identity/v3/role_assignments?user.id=53d524fa52bd42cea09081345ff8d5e6&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:23.611030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32c08f-c842-4440-9708-6533924bfdc2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:23.649550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:23.650227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.650513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.650794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.651509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:23.651817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:23.653389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:23.660816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 383/778] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:23.671123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.785110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ac2a-75aa-4b2f-86a1-77c9574db897 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:23.786714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:23.823585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:23.824264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.824578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.824850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.825651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] RBAC: Adding query filter params (interface=public, service_id=db9b91e1ff49450ab47bf3b139badbf1, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:23.825964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:23.827345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898803d2-d65c-4832-8f63-83f4b47d0590 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:23.837214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 396/779] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:23 2022] GET /identity/v3/endpoints?service_id=db9b91e1ff49450ab47bf3b139badbf1&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:23.842711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:23.855721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:23.947693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf7fa2-9f88-4088-a29d-939434353ced None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.948209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf7fa2-9f88-4088-a29d-939434353ced None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.948319 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf7fa2-9f88-4088-a29d-939434353ced None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.949981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 397/780] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:23.956101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:23.956426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:23.956692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:23.974535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:23.975755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:23.987644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9ff7a-bc24-4614-91a4-a27df7d2e8fc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:24.046357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:24.047411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:24.047918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:24.048458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:24.049716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:24.051120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:24.063431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 384/781] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:23 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:24.083490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 398/782] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:24.113647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:24.238456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:24.288193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589f65c-af71-4172-b434-073d36f88d86 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:24.328919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:24.330660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:24.331169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:24.331649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:24.342639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:24.350605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:24.351547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:24.352454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:24.355062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 385/783] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:24.363628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:24.551623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:24.558522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57127a31-b004-4d45-bbf1-fb537d800cd3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:24.610579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:24.611809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:24.612152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:24.612710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:24.613691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:24.614005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:24.615423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:24.629196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 399/784] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:24.634513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:24.774355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea441f-72bc-4745-a704-14e968ef5512 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:24.805671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:24.816231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:24.817381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:24.817809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:24.818229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:24.829991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:24.831749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:24.850757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:24.852361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:24.853492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:24.853492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:24.853492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:24.853492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:24.855514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 386/785] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:24.861277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.045938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253899a-e7f7-40c4-9944-86ed338e56d3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:25.090434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:25.091018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.091301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.091579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.092297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:25.092630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:25.093994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6694d0-c282-4e0c-9a0f-375497d424c7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:25.110610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 400/786] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:25.116888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.212673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148470-e3a4-4694-83b8-9c78bdaecaca None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.212966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148470-e3a4-4694-83b8-9c78bdaecaca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.213259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148470-e3a4-4694-83b8-9c78bdaecaca None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.214840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 401/787] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:25.225394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.225706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.225987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.252641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:25.253850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:25.281152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7002c-91c4-4363-a03d-c28f517f781d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:25.284704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:25.340902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57887101-106a-4803-bc96-db0af3251635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:25.341632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57887101-106a-4803-bc96-db0af3251635 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.341919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57887101-106a-4803-bc96-db0af3251635 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.342199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57887101-106a-4803-bc96-db0af3251635 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.359200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57887101-106a-4803-bc96-db0af3251635 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:25.360732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57887101-106a-4803-bc96-db0af3251635 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-57887101-106a-4803-bc96-db0af3251635 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:25.382448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:25.383488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:25.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 387/788] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:25.392175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57887101-106a-4803-bc96-db0af3251635 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.430300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 402/789] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:25.453155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:25.477678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.517279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57887101-106a-4803-bc96-db0af3251635 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:25.557949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:25.558628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.558917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.559198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.561024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:25.561355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:25.562874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:25.583838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 388/790] 10.222.0.16 () {56 vars in 1135 bytes} [Thu May 26 13:31:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:25.588838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.714257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cadb9-8c40-4315-93e6-7c3efc561cfb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:25.777088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:25.778347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.778854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.779340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.780207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:25.782142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:25.796494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f291-f322-426a-98b5-6e264e495092 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:25.803722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:25.812965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:25.813983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 13:31:25.815651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:25.820878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 403/791] 10.222.0.16 () {56 vars in 1129 bytes} [Thu May 26 13:31:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:25.826289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.845037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:25.894162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:25.894982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:25.895267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:25.895545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:25.928482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:25.928482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:25.961993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:25.962963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:25.963849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:25.963849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:25.963849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:25.963849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:25.968356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 389/792] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:25.973980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:25.988009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eddeb-8071-451f-b205-37957f686ed7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.078984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.079758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.080618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.081041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.081644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:26.086811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.088326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.102182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 404/793] 10.222.0.16 () {56 vars in 1124 bytes} [Thu May 26 13:31:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:26.107920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.150968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae152514-3da7-4b6a-bf5c-0a6c3a3c90e5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.189151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.190139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.190426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.190708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.191493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:26.191811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.193255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b017e4c-c393-461b-bb0a-1a96fce366c3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.210766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 390/794] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:26.212220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a0996-90ab-4bf8-8f53-48b98f4d3982 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.212567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a0996-90ab-4bf8-8f53-48b98f4d3982 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.212856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a0996-90ab-4bf8-8f53-48b98f4d3982 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.214414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 391/795] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:26.215127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a0996-90ab-4bf8-8f53-48b98f4d3982 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.222586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:26.237250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:26.260849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d0e16-f9d2-49a3-8191-5da451662e90 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.311540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.312754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.313814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.314104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.315383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:26.315702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.317163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131b96e-0ee1-4ac6-be6e-eeece9322c63 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.323708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 405/796] 10.222.0.16 () {56 vars in 1124 bytes} [Thu May 26 13:31:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:26.325256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.325606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.325892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.344481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a0996-90ab-4bf8-8f53-48b98f4d3982 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.350702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:26.353624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:26.390648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.391969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.393060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.393824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.403510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.405355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:26.426167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:26.427166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:26.428115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:26.430470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 392/797] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:26.431350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.464418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 406/798] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:26.465576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.527567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:26.553528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:26.565564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd625c7-3ca1-446e-86b9-ebba25b9a05f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.587814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5356a-20fd-42ab-acc2-8f3e1f1f1445 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.618789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367aec-f6e2-486e-b3e2-d2f6c7c4ba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.619474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367aec-f6e2-486e-b3e2-d2f6c7c4ba8c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.619755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367aec-f6e2-486e-b3e2-d2f6c7c4ba8c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.620033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367aec-f6e2-486e-b3e2-d2f6c7c4ba8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.620871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367aec-f6e2-486e-b3e2-d2f6c7c4ba8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.622553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367aec-f6e2-486e-b3e2-d2f6c7c4ba8c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.630920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.631941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.632481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.632947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.634004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:26.635179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.636837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82325441-ff4a-4353-a438-f890beddf2be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.656337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 393/799] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:26.656337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5f69c-8117-42e5-a4b5-1430372e9f52 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.656337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5f69c-8117-42e5-a4b5-1430372e9f52 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.656337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5f69c-8117-42e5-a4b5-1430372e9f52 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.656337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 394/800] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:26.656337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5f69c-8117-42e5-a4b5-1430372e9f52 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.662485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 407/801] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:26.664158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.664633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.665151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.687819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:26.689228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:26.786652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 408/802] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:26.787469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.836142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5f69c-8117-42e5-a4b5-1430372e9f52 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.868951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:26.869162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:26.877330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.878126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.878411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.878685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.889936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:26.898157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:26.900044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 395/803] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:26.901749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:26.926651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e6342-4cc6-4d75-965d-c9d9d8e3e14f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:26.972465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68c99e-a7f2-4933-b5fc-7c5dfee76b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:26.973456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c99e-a7f2-4933-b5fc-7c5dfee76b1a None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.973822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c99e-a7f2-4933-b5fc-7c5dfee76b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.974232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c99e-a7f2-4933-b5fc-7c5dfee76b1a None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/e4e2c8a980714257b8df1d486d446eeb/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.976133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68c99e-a7f2-4933-b5fc-7c5dfee76b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=e4e2c8a980714257b8df1d486d446eeb, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:26.977656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68c99e-a7f2-4933-b5fc-7c5dfee76b1a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:26.992267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 409/804] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:26 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/e4e2c8a980714257b8df1d486d446eeb/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:26.993606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d340e-4b1b-4245-b053-ad188b52048d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:26.993898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d340e-4b1b-4245-b053-ad188b52048d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:26.994184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d340e-4b1b-4245-b053-ad188b52048d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:26.999790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 410/805] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:26.999790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d340e-4b1b-4245-b053-ad188b52048d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.039839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508df9b-6255-4a73-a601-8bd3baaf4a26 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.098214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86f1c0-8a1d-4d69-a985-52619198640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.099670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f1c0-8a1d-4d69-a985-52619198640f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.100167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f1c0-8a1d-4d69-a985-52619198640f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.100988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86f1c0-8a1d-4d69-a985-52619198640f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.116591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86f1c0-8a1d-4d69-a985-52619198640f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fb86f1c0-8a1d-4d69-a985-52619198640f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:27.128140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:27.129220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:27.130055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:27.133022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 396/806] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:27.134149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.134434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.148186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d340e-4b1b-4245-b053-ad188b52048d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.155719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:27.156995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:27.170710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:27.196136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.196847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.198441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.198441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.198441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:27.198441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.199841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.210611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 411/807] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:27.211487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.222491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:27.256582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 397/808] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:27.257382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.357107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e08e8c-85e4-4ddf-9364-bcdbc68134fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.388118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7a8d-22ee-4966-bc2f-7104104417a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.405510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.406437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.406721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.407385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.411595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:27.411933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.413901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.426170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.426739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 412/809] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:27.427215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.427587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.427715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.428064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.441944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.443631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:27.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:27.465208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:27.466106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:27.466106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:27.466106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:27.466106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:27.466598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 398/810] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:27.472117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.487525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:27.516678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:27.559886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8427667-cdea-4f5f-9197-fc33c8b4150a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.604708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e51dc-14ab-4854-b3b8-f64ed15d198f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.608235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.609344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.609805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.610246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.611593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.613149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:27.630416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:27.631715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 13:31:27.632796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:27.636541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 413/811] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:27.637595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.651476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.652428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.652827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.653382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.666248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.667828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:27.687693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:27.688869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:27.690248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:27.690248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:27.690248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:27.690248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:27.696220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 399/812] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:27.696220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.794755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f164e-f8a4-4e83-8291-dc472b960213 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.802800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:27.840783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:27.846840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.847739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.848130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.848602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.849981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:27.850405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.852126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.868250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a87c59-c9d8-4a5f-967d-f64b8e845f92 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:27.871960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 414/813] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:27.873246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.931184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:27.932719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:27.933589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:27.934323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:27.935231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:27.935873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:27.937782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:27.945502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 400/814] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:27.947212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:27.989984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568ad14-a178-4ea0-87e6-f5190d4b2b3e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.025828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.026692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.027041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.027421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.028245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:28.028772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.030400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.048404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 415/815] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:28.048404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.072664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.084089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a243a9-52e9-4f97-9520-bfb7b849255e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.121297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.122009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.122285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.122562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.130628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.132008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:28.148821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:28.149893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:28.150846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 401/816] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:31:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:28.152030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.165774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.193919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c3578-ab08-44a5-8d98-b690261030d9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.231593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.232273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.232585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.232862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.233578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:28.233886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.235278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.240975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 416/817] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:28.241881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.294397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3b375-4037-4aab-a89c-195b922bcb5d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.331502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.332437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.332646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.332927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.340899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.342496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:28.359931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:28.361561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:28.362532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:28.364012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.366618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 402/818] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:28.367520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.383365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836912b0-c23b-4f12-9d4e-1fdffe63925b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.432155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.433160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.433629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.434085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.435076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:28.435544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.437178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.441074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.459833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 417/819] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:28.461133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.479904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299d80a-9735-4a90-9d12-a75c104f933f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.517684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.518541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.518903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.519254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.520261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.521929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.546545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 403/820] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:28.547685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.585157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d22df6-6220-400e-b967-caa54124f11f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.639260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.640646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.641284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.641930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.644416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:28.646235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.658771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.667537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a69664-4a93-4323-9170-964b21c1be1b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.690750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.697036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.706926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 418/821] 10.222.0.16 () {56 vars in 1149 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:28.713387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.714463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.715254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.715596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.715882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:28.798059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:28.799788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:28.940193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622af9-858a-4735-a9f7-7f1df2c8e556 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:28.942822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 404/822] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:31:28.948150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:28.979132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:28.994207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:28.995442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:28.995988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:28.996574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.010668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.012679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:29.034510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:29.035520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:29.036493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:29.036493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:29.036493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:29.036493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:29.038897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 419/823] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:29.047608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.076696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80515d-7cce-4c50-a934-69663354034b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.120642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.121542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.121878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.122156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.122805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.137059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.138911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:29.159285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:29.160286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:29.161201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:29.161201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:29.161201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:29.161201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:29.163469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 405/824] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:29.169116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.185507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6027b7-b785-4873-acc2-2b458e448acd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.228330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.229017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.229402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.229540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.230337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:29.230568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.231951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b53e4d-0832-4117-80ee-bc438ce1da0a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.244973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 420/825] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:29.246103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2d50f-e12a-4257-a625-76e3df80c29d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.246398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2d50f-e12a-4257-a625-76e3df80c29d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.246690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2d50f-e12a-4257-a625-76e3df80c29d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.248035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 421/826] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:29.250171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2d50f-e12a-4257-a625-76e3df80c29d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.257412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.283735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0427-62e4-45df-baae-0a0c4ba850c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.322399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.323059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.323338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.323613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.324349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:29.324657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.326034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71905fd-591d-4cb5-adfe-68baf5c91e44 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.338005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 406/827] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:29.339233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c17c-c548-4e2e-86ff-f73930bd143b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.339520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c17c-c548-4e2e-86ff-f73930bd143b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.339796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c17c-c548-4e2e-86ff-f73930bd143b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.358382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6db4c17c-c548-4e2e-86ff-f73930bd143b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:29.359599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6db4c17c-c548-4e2e-86ff-f73930bd143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:29.367848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2d50f-e12a-4257-a625-76e3df80c29d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.381565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.404192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.405011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.405290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.405565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.412530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.413972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:29.428044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:29.429032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:29.429949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:29.430384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 407/828] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:29.430384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfed07-58a2-4bf5-87a9-27152ee35efe None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.430524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 422/829] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:29.430591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfed07-58a2-4bf5-87a9-27152ee35efe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.430591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfed07-58a2-4bf5-87a9-27152ee35efe None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.431168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.433065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 408/830] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:29.443220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfed07-58a2-4bf5-87a9-27152ee35efe None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.511586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.534039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bba60-360f-4725-bc77-24a8e0694fa5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.549672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfed07-58a2-4bf5-87a9-27152ee35efe None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.567838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.568667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.568957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.569237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.576213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.577686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.584147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.584834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.585119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.585395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.586174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:29.586489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.587917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-82bea9b3-a5e1-4a4c-97a0-be83cd61d74e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:29.591891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:29.593111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:29.594027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 423/831] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:29.594516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.594516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.594781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.600883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 409/832] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:29.601719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.612208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:29.613427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:29.639843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.681345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 424/833] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:29.681927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.709755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bf6e2-7cf5-4f58-b1f4-af0d26622e64 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.735358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.744898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.745690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.745971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.746248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.746893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.748270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:29.760910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:29.762124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:29.763241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 410/834] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:29.764043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.789119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d2e44-b1f6-4c19-a0be-58e1679ed0ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.824031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.824720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.825011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.825291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.826080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:29.826398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.827845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.839908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 425/835] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:29.840756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.865915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.872884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd109f-827a-4e44-be84-07eb3f80f0d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.908697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.909609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.909907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.910190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/53d524fa52bd42cea09081345ff8d5e6/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:29.925489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=53d524fa52bd42cea09081345ff8d5e6, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:29.926950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:29.940034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 411/836] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:29 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/53d524fa52bd42cea09081345ff8d5e6/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:29.940872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:29.949981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5281a-5959-45db-8ab8-4aaec1033c03 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:29.988969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:29.997270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:29.998127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:29.998556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:29.998677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.013065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:30.021459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:30.022503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 426/837] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:30.024009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.072874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb5b78-b6c4-4bbb-9ff8-7c84a2623f07 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.113386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.114079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.114361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.114652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.115360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:30.115671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.116409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:30.117317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.127829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 412/838] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:30.127829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.151554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c020ff-25f7-4ae4-8632-75f5fdaf7aac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.195154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.195998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.196291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.197020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.198142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=535f23db24704422a4128ff09f1cc284) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:30.198465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.200196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.208727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 427/839] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/role_assignments?user.id=535f23db24704422a4128ff09f1cc284&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:30.209603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.251225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18440d-45af-408c-8308-216b1bf08f66 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.255186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:30.300003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.301934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.302489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.303132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.304265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:30.304799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.307283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.320725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 413/840] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:30.328692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.374005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fdc7-fb23-4eee-81db-7613d7ea93e8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.421882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.423011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.423303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.423625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.424557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:30.424880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.426267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.440354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 428/841] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:30.462908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da5bfd-3b45-461b-8ce1-a5086868f240 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.508452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.509701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.510078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.510453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.524989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.526800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:30.546769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:30.548015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:30.550046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:30.550046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:30.550046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:30.550046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:30.552213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 414/842] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:30.563277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.571457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:30.674770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395d4fd-7958-4623-946a-969908b0a2e5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.712002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.712710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.713055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.713283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.714012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:30.714322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.715701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.729087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 429/843] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:30.734596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.755085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:30.842017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7bcd-a2b1-4a11-b23f-2e96476d4c99 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:30.878586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:30.879386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:30.879668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:30.879945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:30.887923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:30.889420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:30.904197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:30.905435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:30.906693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 415/844] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:30.914802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:30.998294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:31.033343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c406-b350-4fad-b4e5-354586b1239a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:31.069436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:31.070109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:31.070397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:31.070678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:31.071514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:31.071826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:31.073294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:31.092799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 430/845] 10.222.0.16 () {56 vars in 1135 bytes} [Thu May 26 13:31:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:31.098495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:31.183576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:31.206986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a688-c687-4635-8ee6-5f25362e5f76 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:31.243000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:31.243977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:31.244266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:31.244590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:31.315657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:31.317104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:31.397832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:31.430048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 416/846] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:31:31.437742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:31.545968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582d66a-2f04-45d5-a9e0-cdc32f9767d4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:31.574817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:31.584087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:31.584921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:31.585208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:31.585488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:31.596129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:31.597628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:31.616635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:31.617574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:31.618483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:31.618483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:31.618483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:31.618483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:31.618483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 431/847] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:31.623929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:31.730343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56a10b-476e-4f68-8ad3-80022519b07e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:31.765632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:31.766305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:31.766592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:31.766870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:31.767834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:31.768186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:31.769671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:31.781802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 417/848] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:31.788022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:31.894386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fffeb-6f30-4235-86f6-9a578959feac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:31.931630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:31.932466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:31.932759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:31.933039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:31.940195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:31.941685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:31.956531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:31.957470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:31.958969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 432/849] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:31.965263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:32.006984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:32.070213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d650a-36a7-4853-a5ff-acb2760079ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:32.108803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:32.110297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.111330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.111851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.112879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:32.113404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:32.115117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc476f2a-2622-4b62-9aab-11691dda3f13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:32.130200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 418/850] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:32.136045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:32.186342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:32.257674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e3fb1-da26-43bb-9f92-76611ab73981 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:32.295893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618f170-d467-4307-a405-d10076949f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:32.296595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f170-d467-4307-a405-d10076949f04 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.296885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f170-d467-4307-a405-d10076949f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.297172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f170-d467-4307-a405-d10076949f04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.298002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618f170-d467-4307-a405-d10076949f04 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=e4e2c8a980714257b8df1d486d446eeb) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:32.298355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618f170-d467-4307-a405-d10076949f04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:32.299839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618f170-d467-4307-a405-d10076949f04 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:32.306549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 433/851] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:32 2022] GET /identity/v3/role_assignments?user.id=e4e2c8a980714257b8df1d486d446eeb&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:32.330356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c2d2e-bd65-4f53-8824-8028a6d6d2eb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.330702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c2d2e-bd65-4f53-8824-8028a6d6d2eb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.330926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c2d2e-bd65-4f53-8824-8028a6d6d2eb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.332758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 419/852] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:32.338848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.339159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.339416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.358332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:32.359549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:32.436582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 434/853] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:32.460778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c2d2e-bd65-4f53-8824-8028a6d6d2eb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:32.585230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:32.589223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c2d2e-bd65-4f53-8824-8028a6d6d2eb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:32.632179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5bfd5-0687-4308-b484-4f235f144c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:32.633043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5bfd5-0687-4308-b484-4f235f144c5a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.633328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5bfd5-0687-4308-b484-4f235f144c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.633609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5bfd5-0687-4308-b484-4f235f144c5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.646189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5bfd5-0687-4308-b484-4f235f144c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4cc5bfd5-0687-4308-b484-4f235f144c5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:32.654977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:32.655947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:32.656833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 420/854] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:31:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:32.664476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:32.703754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:32.761814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521272-265f-47b7-80b8-3635ccf6c63e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.762153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521272-265f-47b7-80b8-3635ccf6c63e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.762382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521272-265f-47b7-80b8-3635ccf6c63e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.764021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 421/855] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:32.769924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ef56d-0529-4e8c-ac53-d92140d210a1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.770212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ef56d-0529-4e8c-ac53-d92140d210a1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.770490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ef56d-0529-4e8c-ac53-d92140d210a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.782015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b19f20-78f4-40b3-82a7-6dc9f0c0305a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:32.789691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-215ef56d-0529-4e8c-ac53-d92140d210a1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:32.790911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-215ef56d-0529-4e8c-ac53-d92140d210a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:32.820818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:32.821805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.822089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.822374 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.823118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:32.823423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:32.824852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92bc2b-7642-4a19-b527-0ed52a66148e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:32.833962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 435/856] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:31:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:32.835315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe771d-637a-43a7-9b91-c514f36ead4b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.835603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe771d-637a-43a7-9b91-c514f36ead4b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.835883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe771d-637a-43a7-9b91-c514f36ead4b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.839111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 436/857] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:32.839341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe771d-637a-43a7-9b91-c514f36ead4b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:32.870629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 422/858] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:32.872237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:32.872633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:32.873045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:32.896852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:32.898050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:32.952557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:32.960623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe771d-637a-43a7-9b91-c514f36ead4b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:32.988338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 423/859] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:32.988338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.004390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.005317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.005669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.006016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.009823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:33.018836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.020440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:33.042769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:33.043742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:33.044812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:33.044812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:33.044812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:33.044812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:33.044812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 437/860] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:33.045145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.102735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c6acd-2c32-404e-bed5-2bd2bac94a8d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.138335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.139272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.139552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.139846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.140550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.142151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.148510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418178-0465-4c44-a18a-f3967597f5f5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:33.155346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:33.156348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:33.157210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 424/861] 10.222.0.16 () {56 vars in 1132 bytes} [Thu May 26 13:31:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:33.158911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.192337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.193389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.193751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.194046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.194781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.196165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.221147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 438/862] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:33 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:33.222068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.241425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:33.261918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:33.289050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccc766-6372-4dfe-b471-0ac73b9c4cd2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.338236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.338914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.339196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.339472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.340197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:33.340602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.342156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.361249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 425/863] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:33.362562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.378932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505f2e0-4637-4935-a866-0484134204c4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.427846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.429224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.429557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.430003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.431093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:33.431560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.432984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.438689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 439/864] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:33.444458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.484291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c9c29-dee6-4189-8b4c-2b118ec0a57c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.527457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:33.536427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.537553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.537840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.538122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.549799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.551850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.554147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:33.564776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a60d4f-71dc-4877-b973-2a63a80c3fba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:33.575159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:33.576139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:33.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 426/865] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:31:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:33.583195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.614476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.615401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.615689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.616401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.617391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:33.617705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.619909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.626419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 440/866] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:33.706789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527649a-5595-4200-97a9-fdf3e0101dc5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.750240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.750858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.751144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.752072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.752988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:33.753300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:33.754830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:33.776922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 427/867] 10.222.0.16 () {56 vars in 1149 bytes} [Thu May 26 13:31:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:33.783073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:33.915667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaaac2-c662-4418-b761-76579d8babb8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:33.952405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:33.953261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:33.953627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:33.953997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:33.968703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:34.026651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:34.028022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:34.133455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 441/868] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:33 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:31:34.138972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:34.198982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:34.250380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79970b-f1e9-4284-a8ce-afdbef4545dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:34.287334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:34.288440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:34.288875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:34.289303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:34.300937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:34.302604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:34.321195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:34.322382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:34.323516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:34.323516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:34.323516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:34.323516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:34.325416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 428/869] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:34.336338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:34.367531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:34.437741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465777e0-cfb0-4e24-9214-c0fe6a0487e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:34.475151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:34.475823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:34.476106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:34.476473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:34.477196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:34.477503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:34.478868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:34.491644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 442/870] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:34.497061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:34.613349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b618f-8b52-4828-8454-1fb5e5d1952d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:34.623479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:34.653599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:34.654697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:34.655199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:34.655676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:34.663712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:34.665414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0bcbe92a-ce3b-43bb-8ca5-14b0d1e56954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:34.680095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:34.681052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:34.681976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:34.684114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 429/871] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:34.689544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:34.750504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769b65-9c2d-4509-911d-2e470d1ca4c3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:34.751174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769b65-9c2d-4509-911d-2e470d1ca4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:34.751722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769b65-9c2d-4509-911d-2e470d1ca4c3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:34.753660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 430/872] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:34.759157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:34.759743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:34.760233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:34.777472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:34.778989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:34.808506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a7aa0-e3f4-47f0-b1b4-6ad49e22755b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:34.820739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:34.853545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:34.854662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:34.854952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:34.855244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:34.856112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:34.856531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:34.861580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:34.862080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 431/873] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:34.873540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 443/874] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:34.879724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:34.881251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:35.003204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:35.010767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec381f9b-dcd0-4a1c-af22-0986e95f9a10 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:35.011668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc1cce-b4c0-48c9-a59e-955e0245acb0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:35.059465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:35.060842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.061361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.061892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.062778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:35.065644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:35.074780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:35.075469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.075758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.076042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.076915 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=53d524fa52bd42cea09081345ff8d5e6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:35.077238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:35.078700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:35.081579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:35.082579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:35.083766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:35.092213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 444/875] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:35.093498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:35.102554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 432/876] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:34 2022] GET /identity/v3/role_assignments?user.id=53d524fa52bd42cea09081345ff8d5e6&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:35.147686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:35.247770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81281df-9560-46d7-8d72-663721773be3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:35.288523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:35.289207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.289498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.290505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.290505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:35.290802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:35.292250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a07510-d360-416a-b7c4-8ec9ea7806e9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:35.304578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 445/877] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:35.310201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:35.443015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebebd24-cb5a-4d1b-b8ea-4e7369342bf7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:35.480969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:35.481650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.481930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.482205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.482910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:35.483216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:35.484616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fe89f-30ef-4d06-9c38-6341ab75a13c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:35.490248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 433/878] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:35.623116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8424cf-e6cc-4b50-b187-a6bd09984a80 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.623411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8424cf-e6cc-4b50-b187-a6bd09984a80 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.624062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8424cf-e6cc-4b50-b187-a6bd09984a80 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.625797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 446/879] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:35.633061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.633061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.633061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.653171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:35.654530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:35.681101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:35.739412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 434/880] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:35.761915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8424cf-e6cc-4b50-b187-a6bd09984a80 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:35.877476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:35.897448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8424cf-e6cc-4b50-b187-a6bd09984a80 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:35.934749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:35.935563 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:35.935848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:35.936130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:35.936841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:35.938349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f39941a0-bde1-408a-9efa-d5bc155b38b0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:35.950735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:35.951743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:35.952614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 447/881] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:35.959089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:36.034564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbafe4-3079-4f61-87c0-ed7c770c6a8d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.034868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbafe4-3079-4f61-87c0-ed7c770c6a8d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.035134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbafe4-3079-4f61-87c0-ed7c770c6a8d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.036836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 448/882] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:36.042143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.042437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.042723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.065516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:36.066759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:36.080587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d79520-7396-4a63-8fd5-ac488a726807 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:36.127165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:36.128694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.129188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.129726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.131096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:36.131640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:36.133814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:36.141480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 435/883] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:36.148019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:36.160409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 449/884] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:36.176743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:36.179044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:36.281368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:36.292542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703e5ac-695a-40b5-bf04-46e155dd20b8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:36.308769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa525-025c-4946-8ff6-c00d25222b60 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:36.339030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:36.339810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.340103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.340412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.341131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:36.341433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:36.343585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:36.351573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:36.352590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.353060 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.353673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.354617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:36.355420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 436/885] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:36.356804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:36.356928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:36.384469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 450/886] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:36 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:36.453428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:36.470907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41290657-2263-4121-bf80-5d262dfb3b16 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:36.516342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:36.517579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.517866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.518151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.518949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] RBAC: Adding query filter params (interface=public, service_id=909c4f31148e4417bce35488ef23b3f5, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:36.519267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:36.520721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200ee13-260d-43aa-8f76-14473ec5d4ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:36.532014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 437/887] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:36 2022] GET /identity/v3/endpoints?service_id=909c4f31148e4417bce35488ef23b3f5&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:36.534996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:36.646824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106513-46ba-42ee-b7b5-30c519271363 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:36.693374 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9a986-e9f3-4371-9963-83a3f5f8cece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:36.694088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9a986-e9f3-4371-9963-83a3f5f8cece None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:36.694359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9a986-e9f3-4371-9963-83a3f5f8cece None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:36.694634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9a986-e9f3-4371-9963-83a3f5f8cece None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:36.695403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9a986-e9f3-4371-9963-83a3f5f8cece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:36.696855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9a986-e9f3-4371-9963-83a3f5f8cece None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:36.708448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 451/888] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:36 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:31:37.792698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097be4e-cc3e-412d-bbc0-1a77975ea8a6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:37.793246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097be4e-cc3e-412d-bbc0-1a77975ea8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:37.793246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097be4e-cc3e-412d-bbc0-1a77975ea8a6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:37.794861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 438/889] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:37.800405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:37.800812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:37.801009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:37.806143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:37.822166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:37.823067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d4ca4-6d96-444a-9ba4-e6bad50902ab None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:37.823358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d4ca4-6d96-444a-9ba4-e6bad50902ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:37.823640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d4ca4-6d96-444a-9ba4-e6bad50902ab None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:37.824409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:37.825337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 439/890] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:37.831765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:37.832075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:37.832353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:37.850025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:37.851219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:37.906684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 452/891] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:37.934340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:37.942902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 440/892] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:37.965675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.056687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:38.058039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fac6f-abdd-4b9b-aaf4-b5643f585edf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:38.091837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a943ae-7926-4bfd-a04b-0891b151389d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:38.092577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:38.104660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:38.106153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.106439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.106721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.122062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:38.129503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:38.130168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.130451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.130730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.131459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:38.132112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:38.133063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:38.133932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:38.134567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:38.138500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 453/893] 10.222.0.16 () {56 vars in 1120 bytes} [Thu May 26 13:31:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:38.139344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.160666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 441/894] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:37 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:31:38.260551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f58c1-136c-4c0c-89ff-488384211cdc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:38.305405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:38.306100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.306382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.306659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.307468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:38.307721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:38.309169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:38.319926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 454/895] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:31:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:38.325582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.436701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8f0c4-51a8-48aa-9e38-7466e5055aee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:38.473017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:38.473829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.474167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.474452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.476557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:38.486971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:38.488921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:38.508695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:38.509654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:38.510547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:38.510547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:38.510547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:38.510547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:38.510547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 442/896] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:38.516684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.577899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:38.642815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb194c81-83c3-4dd4-a7b1-1e73928dff78 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:38.682095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:38.683150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.683676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.684154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.685129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:38.685660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:38.687257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fff64-694f-491c-9d67-c853236bf989 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:38.703081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 455/897] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:38.709575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.781775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dabff-1591-4e52-bc01-3f2b5e3c5ccc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.782136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dabff-1591-4e52-bc01-3f2b5e3c5ccc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.782300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dabff-1591-4e52-bc01-3f2b5e3c5ccc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.784439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 456/898] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:38.790247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.790874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.791399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.811330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:38.813442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:38.834020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549efd7-4115-435d-abfe-3cf6d4952910 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:38.881120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:38.882189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:38.882475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:38.882754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:38.893044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:38.894829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:38.900473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 457/899] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:38.910620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:38.911598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:38.912592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:38.915212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 443/900] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:31:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:38.921163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.921413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:38.975156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:38.977993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.039934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231ee66-b758-487b-978a-d7120390ffc5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.042282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4c2e-743d-42c8-a37f-c223aa02a769 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.077006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.077821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.078032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.078146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.078369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.078683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.078967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.079062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.079302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.080392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:39.080709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.080869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.082125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea7542-f01d-4ba7-b69e-fe6ef9b64ac9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e998456c-4d2f-4a2d-b3cd-d5357f5b0ce8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:39.094565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:39.096262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.097125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 444/901] 10.222.0.16 () {56 vars in 1132 bytes} [Thu May 26 13:31:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:39.097965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47eec-d406-4ff6-b39e-df56b11a2cdc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.103831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47eec-d406-4ff6-b39e-df56b11a2cdc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.103831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47eec-d406-4ff6-b39e-df56b11a2cdc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.103831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 445/902] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:39.104507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47eec-d406-4ff6-b39e-df56b11a2cdc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.107808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 458/903] 10.222.0.16 () {56 vars in 1149 bytes} [Thu May 26 13:31:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:39.109146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.109523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.109864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.127297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:39.128651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:39.198918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 459/904] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:39.199770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.236645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47eec-d406-4ff6-b39e-df56b11a2cdc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.237077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.248799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.275294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.275948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.276233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.276542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.277248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:39.277559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.279100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.285843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 446/905] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:39.286672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.323045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3630f8-4682-4f8b-bd40-735682914978 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.359050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.359613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.359894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.360172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.406590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d0370-e4dd-4546-a1d2-336328428367 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.445006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.446470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.453582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.455188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.455488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.455772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.456461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.459002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.475088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:39.476178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:39.477608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.483662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 447/906] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:39.484916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.531975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.559726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 460/907] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:31:39.560893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.618748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abadf6a-f3fc-4461-8554-e09019c274cd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.657027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.657847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.658131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.658415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.659124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:39.659468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.660921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.666333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 448/908] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:39.667173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.680760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573fd06-6fb4-4b52-b0aa-6a04d24b7705 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.710623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.724270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.724967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.725250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.725526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.726232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:39.726536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.727900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.732854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 461/909] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:39.733669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.795319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd8cd5-4d25-4f9f-a961-3232ba527d35 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.796969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:39.837777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.838610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.838850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.839124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.854446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.855958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.856123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dfdf1-a885-4b55-b418-543c805c9494 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:39.882135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:39.883183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:39.884123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:39.884123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:39.884123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:39.884123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:39.884123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 449/910] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:39.884924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:39.908922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:39.910120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:39.910646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:39.911161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:39.914836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] RBAC: Adding query filter params (interface=public, service_id=4d982e77b50642279c2eb7896b0cc05f, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:39.914836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:39.915517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:39.925503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 462/911] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/endpoints?service_id=4d982e77b50642279c2eb7896b0cc05f&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:39.926918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.004985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:40.020046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee6c9e-8f9b-4043-a22d-51a23484d19f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.055486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5d977-192e-4a04-b022-ff94337dd9ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.059268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.060183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.060692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.061158 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.062078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:40.062574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.064659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.072101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 450/912] 10.222.0.16 () {56 vars in 1114 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:40.073650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.092655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.093428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.093728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.094005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.094736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:40.095642 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.097078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.100467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:40.109885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 463/913] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:40.110853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.199120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0b744-ee1b-42c2-968e-43138546231a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.214371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c1d41-caa7-4fc6-9e47-7d2015f6ecf8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.232249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:40.240786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.241751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.241946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.242196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.243033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.244365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.260360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 451/914] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:40 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:31:40.260360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.266026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e87494-e056-4e24-ad17-4d6dd9715d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.267816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e87494-e056-4e24-ad17-4d6dd9715d74 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.268446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e87494-e056-4e24-ad17-4d6dd9715d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.269001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e87494-e056-4e24-ad17-4d6dd9715d74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.270536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e87494-e056-4e24-ad17-4d6dd9715d74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.273263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e87494-e056-4e24-ad17-4d6dd9715d74 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.312206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 464/915] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:40.404364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:40.430999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c9e2a-5328-483f-8a8f-45f3e9242fc8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.481587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.482669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.483106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.483589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.492937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.494825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:40.512444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:40.513548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:40.514639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:40.517019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 452/916] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:40.518204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c752-b388-4b91-afdd-eb4f5d17c5e9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.518798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c752-b388-4b91-afdd-eb4f5d17c5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.519381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c752-b388-4b91-afdd-eb4f5d17c5e9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.522268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 465/917] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:40.526071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.529270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.529789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.530268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.555498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:40.557230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:40.663520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 466/918] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:40.669847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039473d-3e20-47fc-9647-fe8951d77c2d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.689138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.712422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.713110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.713399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.713682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.714518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:40.714836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.716459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.727765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:40.736657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 453/919] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:40.742661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.836800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc9b58-2ab2-4e91-8cda-296a7bd8dff1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.838390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:40.889146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.890158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.890723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.890993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.892376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.895555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:40.908997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:40.910040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:40.910887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:40.915511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 467/920] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:40.916496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:40.925868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df8483-379a-4667-acf1-80ee8e254cd0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:40.965885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:40.967037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:40.967878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:40.968334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:40.969591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] RBAC: Adding query filter params (role.id=414f73f64d774dec8cafd23a785fd4c7, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=53d524fa52bd42cea09081345ff8d5e6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:40.970094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:40.971923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:40.984791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 454/921] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:40 2022] GET /identity/v3/role_assignments?user.id=53d524fa52bd42cea09081345ff8d5e6&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=414f73f64d774dec8cafd23a785fd4c7 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:41.065313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7125fc-ede4-4290-b080-363fd6dd324b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:41.144398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:41.145458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:41.145889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:41.146368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:41.147317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:41.147792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:41.150543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:41.161464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 468/922] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:41.168389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:41.257253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:41.303219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad6c77-a23c-477c-818d-bef189c9c9d6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:41.341445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:41.342454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:41.342981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:41.343463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:41.344407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:41.345469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:41.347213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:41.353548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 455/923] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:41.359653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:41.395794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:41.480024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab4d4-d187-4ceb-aef4-d7a8d9f29940 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:41.516425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:41.517129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:41.517410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:41.517687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:41.518479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] RBAC: Adding query filter params (interface=public, service_id=399b068625c44a2c9f190fc5f2b26b46, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:41.518792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:41.520171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85568d2c-2836-4e71-841a-be296e6c3801 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:41.526078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 469/924] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:41 2022] GET /identity/v3/endpoints?service_id=399b068625c44a2c9f190fc5f2b26b46&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:41.531581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:41.646518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d6f96-f032-4f6c-b703-cc9f1b99d205 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:41.677971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:41.693569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4bdfe-3de5-45d5-92c5-ad6931edee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:41.694250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4bdfe-3de5-45d5-92c5-ad6931edee95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:41.694539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4bdfe-3de5-45d5-92c5-ad6931edee95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:41.694821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4bdfe-3de5-45d5-92c5-ad6931edee95 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:41.695514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4bdfe-3de5-45d5-92c5-ad6931edee95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:41.696944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4bdfe-3de5-45d5-92c5-ad6931edee95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:41.703724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 456/925] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:41 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:31:42.702157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67eb7-df91-4ad3-a38d-57ba9cf19b46 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:42.702700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67eb7-df91-4ad3-a38d-57ba9cf19b46 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:42.702700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67eb7-df91-4ad3-a38d-57ba9cf19b46 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:42.705168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 470/926] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:42.711615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49540-25ba-45a3-8f84-91fe78720cbd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:42.712137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49540-25ba-45a3-8f84-91fe78720cbd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:42.712240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c49540-25ba-45a3-8f84-91fe78720cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:42.730767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11c49540-25ba-45a3-8f84-91fe78720cbd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:42.732008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11c49540-25ba-45a3-8f84-91fe78720cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:42.812422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 457/927] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:42.829729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67eb7-df91-4ad3-a38d-57ba9cf19b46 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:42.914260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9994-d83b-410c-b526-80d80c1c8d7d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:42.914260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9994-d83b-410c-b526-80d80c1c8d7d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:42.914603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9994-d83b-410c-b526-80d80c1c8d7d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:42.916748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 458/928] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:42.927164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:42.927783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:42.928381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:42.936215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:42.950125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67eb7-df91-4ad3-a38d-57ba9cf19b46 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:42.953526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:42.955074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:42.987467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:42.988737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:42.989265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:42.989685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:42.990577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:42.992193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:43.006018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:43.007232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 13:31:43.008258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:43.010814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 471/929] 10.222.0.16 () {56 vars in 1132 bytes} [Thu May 26 13:31:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:43.017796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.042436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 459/930] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:43.066910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.118330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:43.139585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d957437-77a0-4553-91f4-2fd47a26784d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.175220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d85e03-0bcd-4ad9-bd24-223c58ab4d3f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.180637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:43.181601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.182170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.182622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.183578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:43.184069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:43.185777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.197826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 472/931] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:43.199177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.212682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:43.213121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.213405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.213683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.214399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:43.215843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.217246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING keystone.server.flask.common [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 13:31:43.230207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:43.237926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 13:31:43.237926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 13:31:43.237926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [parameters: {'id': 'af5d5b6885e74b20a72bb8e5f14176ce', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 13:31:43.237926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id af5d5b6885e74b20a72bb8e5f14176ce.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id af5d5b6885e74b20a72bb8e5f14176ce. May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:31:43.245636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 13:31:43.246559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 26 13:31:43.247537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 13:31:43.248460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 13:31:43.255965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:31:43.258868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': 'af5d5b6885e74b20a72bb8e5f14176ce', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.261063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:43.262985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id af5d5b6885e74b20a72bb8e5f14176ce. May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 460/932] 10.222.0.16 () {60 vars in 1140 bytes} [Thu May 26 13:31:43 2022] POST /identity/v3/projects => generated 262 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:43.265242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.358794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f658b81-a737-464b-8c8d-7a1c6d79959a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.398132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:43.398805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.399084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.399371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.400064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:43.400415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:43.401847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.410259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59a825-1b5c-4bb6-a67f-0db003e93cf8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.412313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 473/933] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:43.413182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.420120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:43.451649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:43.452499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.452782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.453054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.461576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:43.463307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9d8edeff-88fe-42b6-bb92-28288bd55da2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:43.478530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:43.479496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:43.480415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 461/934] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:43.481383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bda21-ca0f-4a8a-84c2-61b51ef95456 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.481480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bda21-ca0f-4a8a-84c2-61b51ef95456 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.481689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bda21-ca0f-4a8a-84c2-61b51ef95456 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.486100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 462/935] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:43.487037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bda21-ca0f-4a8a-84c2-61b51ef95456 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.533231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:43.550857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac8d9a-74e9-48d9-9f7c-e6ff959c3aa9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.592109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a1f2-7289-46f1-b8c2-3244e1b8b82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:43.592806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a1f2-7289-46f1-b8c2-3244e1b8b82a None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.593092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a1f2-7289-46f1-b8c2-3244e1b8b82a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.593370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a1f2-7289-46f1-b8c2-3244e1b8b82a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.594732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a1f2-7289-46f1-b8c2-3244e1b8b82a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:43.596563 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a1f2-7289-46f1-b8c2-3244e1b8b82a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.618249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 474/936] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:43.619560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.619848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.620128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.640581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:43.642808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:43.663743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bda21-ca0f-4a8a-84c2-61b51ef95456 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.716653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:43.717286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:43.717632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:43.717965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:43.718748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:43.719053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:43.721305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:43.733978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 475/937] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:43.745276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 463/938] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:43.786128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:43.922591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:43.963303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdadb2-b518-44f1-95ff-e876b55bc74f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:43.999859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b71725-04cc-42e5-bc57-71098813dcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.000633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b71725-04cc-42e5-bc57-71098813dcea None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.001666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b71725-04cc-42e5-bc57-71098813dcea None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.002025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b71725-04cc-42e5-bc57-71098813dcea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.013251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b71725-04cc-42e5-bc57-71098813dcea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-90b71725-04cc-42e5-bc57-71098813dcea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:44.021661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:44.022623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:44.023485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 476/939] 10.222.0.16 () {56 vars in 1120 bytes} [Thu May 26 13:31:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:44.030740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.059114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.093775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea581f58-6753-4f48-b58a-d8de9e0c5858 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.094062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea581f58-6753-4f48-b58a-d8de9e0c5858 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.094347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea581f58-6753-4f48-b58a-d8de9e0c5858 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.096036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 477/940] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:44.103099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.103393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.103678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.125668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:44.128373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:44.143071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2ae3c-a85a-4dad-9c5b-4f88bcde45b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.187284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.188487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.189154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.189646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.190598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:44.191131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.192763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.202204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 464/941] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:44.208718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.219035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 478/942] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:44.240890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.307306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.324779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.329656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177932f-eff8-474d-bc54-dfd18549dda3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.371133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.372426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.373000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.373551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.376355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf85c6-d811-404f-914e-b503eb83835e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.386643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.388737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:44.408650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:44.409583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:44.410478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:44.410478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:44.410478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:44.410478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:44.412889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 465/943] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:44.416002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.420337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.420337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.420337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.420337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.420337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.421914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:44.439010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:44.440187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:44.441094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 479/944] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:44.449817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.535068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee94d7f-24ac-4ae9-8cb4-1dd4124eb00a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.572938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.573906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.574398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.574862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.575787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:44.576263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.577826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.581819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.590818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ae31-aa96-446b-b391-2f5b7b400d46 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.598560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 466/945] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:44.599750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.627564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.648644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.649390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.649683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.649968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.650692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:44.651006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.652788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.659466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 480/946] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:44.664645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.723468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d77ce-f9cc-4ff0-a8b9-7c5e89cb9371 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.764109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.765530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.766044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.766574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.775204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.777028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:44.796904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:44.798550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:44.799675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:44.806659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384da99-3792-4c26-b0bd-60d8b126c4b6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.807659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 467/947] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:44.808853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.850885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.851478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.851765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.852046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.852798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:44.853111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.854512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.862355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 481/948] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:44.868864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:44.884626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.937156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632433b-a659-4c7b-9319-390333535f95 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:44.943872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:44.979909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:44.980641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:44.980925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:44.981206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:44.982169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:44.982483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:44.983946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:44.999357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc416-c53d-47d2-9f6a-292e78d0732f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:45.011838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 468/949] 10.222.0.16 () {56 vars in 1149 bytes} [Thu May 26 13:31:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:45.013314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:45.055250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:45.055925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:45.056392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:45.056792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:45.057553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:45.059233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:45.088481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 482/950] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:45.166804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99c5f7-ae65-40a2-bb1e-1037eab26d53 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:45.184194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:45.224705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:45.226107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:45.226675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:45.227163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:45.239764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:45.241774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:45.265759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:45.267383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:45.268565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:45.268565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:45.268565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:45.268565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:45.271201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 469/951] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:45.278020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:45.405499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4148172-87fe-47db-924c-c5a9ad3b64a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:45.417735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:45.447879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:45.448613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:45.448914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:45.449204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:45.449963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:45.450302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:45.451778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:45.466576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 483/952] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:45.472210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:45.592597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399a363-b229-4cee-a844-59bc1999723b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:45.630856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:45.631768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:45.632122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:45.632503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:45.640313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:45.642455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:45.658116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:45.659288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:45.660455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:45.664340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 470/953] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:45.673923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:45.789004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e24c1-7c41-4345-99b0-52ca34e4b5a7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:45.825984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:45.826748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:45.826940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:45.827213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:45.827995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:45.828323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:45.830089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63625b6-c5b2-4d85-ae1f-bdbcee59d1a9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:45.843474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 484/954] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:45.849794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:45.878435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:45.976437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d00a-7411-4235-9055-4a28be937115 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.017444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e0d08-7a53-4409-b86f-c6f75d6729ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.018419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e0d08-7a53-4409-b86f-c6f75d6729ec None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.018711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e0d08-7a53-4409-b86f-c6f75d6729ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.018991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e0d08-7a53-4409-b86f-c6f75d6729ec None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/53d524fa52bd42cea09081345ff8d5e6/roles/414f73f64d774dec8cafd23a785fd4c7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.021369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e0d08-7a53-4409-b86f-c6f75d6729ec None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=53d524fa52bd42cea09081345ff8d5e6, role_id=414f73f64d774dec8cafd23a785fd4c7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.022846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e0d08-7a53-4409-b86f-c6f75d6729ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.040425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 471/955] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:31:45 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/53d524fa52bd42cea09081345ff8d5e6/roles/414f73f64d774dec8cafd23a785fd4c7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:31:46.084624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07e8c-d9fb-4d69-86dc-b9ee56573dfb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.084947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07e8c-d9fb-4d69-86dc-b9ee56573dfb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.085208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07e8c-d9fb-4d69-86dc-b9ee56573dfb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.087036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 485/956] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:46.094085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.094391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.094783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.114952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:46.116717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:46.168707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc3ddf-8f59-4b83-a5f2-f3bfdbd87867 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.169354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc3ddf-8f59-4b83-a5f2-f3bfdbd87867 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.169843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc3ddf-8f59-4b83-a5f2-f3bfdbd87867 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.171707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 486/957] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:46.177890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.178203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.178463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.198030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:46.199351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:46.201005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 472/958] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:46.223705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:46.237498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:46.245598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:46.296825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 487/959] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:46.323024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:46.356550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7379dbe-c5bd-4486-b9a1-89b8a4958832 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.392826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.393631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.393919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.394201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.394855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.396243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:46.412712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:46.413851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:46.414713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:46.415460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 473/960] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:46.419558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:46.446126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04ec66-4d36-4997-8930-c4b004af29e3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.488142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.488771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.489164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.489375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.490124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.491571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.492950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING keystone.server.flask.common [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 13:31:46.507068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.sql.core [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 13:31:46.507068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 13:31:46.507068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [parameters: {'user_id': 'e017980669b34cc192e0606df462bd20', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 13:31:46.507068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 13:31:46.517404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() May 26 13:31:46.529905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:31:46.530898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application yield resource May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.rollback() May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.force_reraise() May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise self.value May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application session.commit() May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 13:31:46.531899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._prepare_impl() May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.flush() May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._flush(objects) May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application compat.raise_( May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application flush_context.execute() May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rec.execute(self) May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 13:31:46.534055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 13:31:46.535260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 13:31:46.536159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e017980669b34cc192e0606df462bd20', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:46.537957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 13:31:46.538885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 488/961] 10.222.0.16 () {60 vars in 1131 bytes} [Thu May 26 13:31:46 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:46.539976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:46.562132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd773a-8f8f-4efb-871b-9b175914b0c3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.573322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:46.614672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.616002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.616513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.616936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.617821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:46.618889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.620790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.629936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 474/962] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:46.635560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:46.677149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e121eb-874f-4f1c-85bb-4aa8618f170b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.715907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.716959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.717225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.717563 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.728059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.729800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:46.745724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:46.746910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:46.748034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 489/963] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:31:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:46.753934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:46.778868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdef358-6161-4077-a50a-7b15423133c4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.829917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.830847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.831524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.831979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.834057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:46.834542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.836081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c57bb-4ee3-43a5-9264-34f4e877f002 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.842863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 475/964] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:46.852199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:46.875398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7f8f9-956c-4aa2-88fe-26c9ddbceaa9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:46.932034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:46.932971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:46.933366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:46.933750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:46.934693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:46.935112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:46.936736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871167c2-7330-469c-bcaa-4c753797dde3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:46.981917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 490/965] 10.222.0.16 () {56 vars in 1101 bytes} [Thu May 26 13:31:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:47.614902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c532b-bd80-4004-8064-4bf7fbf7daf4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:47.615509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c532b-bd80-4004-8064-4bf7fbf7daf4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:47.615509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c532b-bd80-4004-8064-4bf7fbf7daf4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:47.617246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 476/966] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:47.622947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:47.623237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:47.623520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:47.645934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:47.647655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:47.731264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 491/967] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:47.755080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c532b-bd80-4004-8064-4bf7fbf7daf4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:47.760274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:47.862830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c532b-bd80-4004-8064-4bf7fbf7daf4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:47.896888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:47.897611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:47.897898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:47.898179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:47.898838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:47.900216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:47.918472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:47.920048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:47.920986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 477/968] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:47.922377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:48.007744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:48.041079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9cd22-5f0f-4e53-b114-fdb299941a7c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:48.081732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:48.082420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.082712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.083252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.083985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:48.084294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:48.085729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:48.096413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 492/969] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:48.097781 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:48.174736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:48.212700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eea469-a680-47be-a964-f6f93f273bab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:48.252727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:48.253411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.253698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.253977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.254717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:48.255031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:48.256481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbd601-f7d9-47aa-ac0c-6d5a1f6fe63a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:48.263081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 478/970] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:48.270111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:48.429306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2bd81-0903-4a13-a9c6-6aee89c10ebb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:48.461343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d3d82-9f0f-4dec-9308-12eab37065d9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.461795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d3d82-9f0f-4dec-9308-12eab37065d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.462143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d3d82-9f0f-4dec-9308-12eab37065d9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 479/971] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:48.470176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.470488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.470744 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.492782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:48.493601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.493892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.494101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:48.494203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.495137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] RBAC: Adding query filter params (interface=admin, service_id=399b068625c44a2c9f190fc5f2b26b46, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:48.495489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:48.496195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:48.497136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:48.498933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:48.510896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 493/972] 10.222.0.16 () {56 vars in 1249 bytes} [Thu May 26 13:31:48 2022] GET /identity/v3/endpoints?service_id=399b068625c44a2c9f190fc5f2b26b46&interface=admin®ion_id=RegionOne => generated 193 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:48.517547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:48.587695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 480/973] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:48.632246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:48.660476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4984be-849f-489e-8b39-4474a93905cf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:48.707630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:48.708717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.709001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.709280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.710015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:48.711552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:48.728601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 494/974] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:48 2022] GET /identity/v3/services => generated 1497 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:31:48.733473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:48.796024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177090a-d53c-4b72-9e0d-ff1011166de0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:48.842934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:48.843778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:48.844145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:48.844732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:48.857084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:48.866007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:48.867159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:48.868021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 481/975] 10.222.0.16 () {56 vars in 1120 bytes} [Thu May 26 13:31:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:48.873459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:48.961160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:49.003375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f8434-6578-4332-81d2-08d0361bc08a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:49.039488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:49.040162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.040492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.040770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.041506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:49.041814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:49.043181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd202-591a-474d-b33e-6ecbcef45214 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:49.052536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 495/976] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:31:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:49.057881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:49.112587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f327d-06e4-44bc-8106-24264211bae3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.112874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f327d-06e4-44bc-8106-24264211bae3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.113162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f327d-06e4-44bc-8106-24264211bae3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.114673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 496/977] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:49.119714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.120003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.120282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.138141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:49.139343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:49.165842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a9a9c-857c-4b38-87f4-7dd907a146d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:49.191241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:49.205521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:49.206344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.206625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.206906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.214971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 497/978] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:49.222723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:49.224197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:49.237827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:49.242633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:49.243542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:49.244667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:49.244667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:49.244667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:49.244667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:49.244667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 482/979] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:49.249456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:49.303519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:49.359136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a3d2-56d2-41c4-886e-7b1b63f499fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:49.362665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2c0af-7af0-4a37-934d-df5cad640daa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:49.399524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:49.400210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.400347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:49.400528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.400817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.401544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.401672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:49.402117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.402622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.402946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:49.403709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:49.404216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:49.405934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:49.419580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 498/980] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:49 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:31:49.420918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 483/981] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:49.429795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:49.563075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b43e61-2915-4ca1-a03e-a062e0559013 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:49.577262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:49.608573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:49.609412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.609695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.610031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.620040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:49.621574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:49.639711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:49.641035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 13:31:49.642199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:49.644696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 499/982] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:31:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:49.647006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:49.687500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:49.810382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77de44b-5825-4b9b-8890-cce28489ea6d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:49.848565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:49.849166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:49.849454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:49.849736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:49.850591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:49.850906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:49.852412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:49.885182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 484/983] 10.222.0.16 () {56 vars in 1149 bytes} [Thu May 26 13:31:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:49.886973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:50.010244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163b522-1d57-4707-ac4c-81110b259c67 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:50.052081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:50.052792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:50.053150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:50.053438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:50.103994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:50.137682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:50.139263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:50.249419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 500/984] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:49 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:31:50.259073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:50.380978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:50.391238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160cab5-0489-473f-82a1-53b3576a2f10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:50.433695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:50.434533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:50.434821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:50.435102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:50.446794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:50.448291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:50.467718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:50.469102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:50.470475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:50.470475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:50.470475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:50.470475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:50.470475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 485/985] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:50.476793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:50.551010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:50.588666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b09e54-8747-42f0-9c54-dc694d9e8f2b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:50.628278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:50.628980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:50.629426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:50.629587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:50.630338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:50.630708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:50.632229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:50.650879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 501/986] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:50.652351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:50.767152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1221b6-fe8d-4d07-91ac-1b1367f09f05 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:50.809522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:50.810320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:50.810598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:50.810874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:50.818339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:50.819797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:31:50.834491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:50.835453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:50.836805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 486/987] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:50.841730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:50.963672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509245-6a51-4dc6-aa38-98a11925509e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:50.994977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:51.002264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:51.002980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.003284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.003585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.004463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:51.004827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:51.006397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdcbd7-1469-470b-8f56-c05b966e0ea1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:51.023858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 502/988] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:31:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:51.025853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:51.030441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:51.053634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e79a9-b49a-42ae-8463-76b6d9da12f1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.053992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e79a9-b49a-42ae-8463-76b6d9da12f1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.054203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e79a9-b49a-42ae-8463-76b6d9da12f1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.055574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 503/989] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:51.060770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.061106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.061329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.078989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:51.080179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:51.137275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bb394-fe5a-4623-9beb-b080db2e8af4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:51.155622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 504/990] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:51.173999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:51.174568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.174852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.175131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.175961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] RBAC: Adding query filter params (role.id=414f73f64d774dec8cafd23a785fd4c7, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=53d524fa52bd42cea09081345ff8d5e6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:51.176271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:51.177786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:51.179388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:51.189119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 487/991] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:31:51 2022] GET /identity/v3/role_assignments?user.id=53d524fa52bd42cea09081345ff8d5e6&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=414f73f64d774dec8cafd23a785fd4c7 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:51.309124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac994da2-2db1-41b3-a5c8-81bc617f3b62 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:51.336525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:51.356710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:51.357588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.357871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.358152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.358851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:51.361082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:51.374449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:51.376136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:51.385406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 505/992] 10.222.0.16 () {56 vars in 1126 bytes} [Thu May 26 13:31:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:51.386854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:51.470304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:51.509509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf131fd6-f75e-4bd7-90c1-9b1bb960182f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:51.549856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:51.550525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.550807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.551084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.551787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:51.552100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:51.553534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:51.563102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 488/993] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:51.564965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:51.678742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f9e1c-862a-483a-aca2-91a43d387251 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:51.713421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:51.714096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.714381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.714662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.715369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:51.715678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:51.717084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390571c-29ef-497c-84b0-b849b3c3ea92 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:51.726830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 506/994] 10.222.0.16 () {56 vars in 1122 bytes} [Thu May 26 13:31:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:51.728137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:51.775388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d481eb-0b8b-464f-ad6b-6d0355292f7d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.775719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d481eb-0b8b-464f-ad6b-6d0355292f7d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.775950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d481eb-0b8b-464f-ad6b-6d0355292f7d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.778122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 507/995] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:51.783547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.783834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.784112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.800807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:51.801984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:51.843059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1740d9-b5fb-4d11-ab43-aa1e085316e2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:51.849356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:51.881830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 508/996] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:51.883421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:51.896060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:51.896060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:51.896060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:51.896060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:51.896499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:51.896499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:51.907766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:51.924339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 489/997] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:52.026993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7bb9f-c06c-4199-b158-5eb92b0be8fe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:52.072988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:52.073814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:52.074096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:52.074475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:52.075019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:52.076431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:52.089901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:52.091128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:52.092027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:52.096638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 509/998] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:31:52.098170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:52.200014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12ab87-1df0-4f6a-bef5-8bf593d3637e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:52.250566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:52.251265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:52.251730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:52.252013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:52.253077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:52.253624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:52.255924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:52.268161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 490/999] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:52.269817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:52.311586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:52.376166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd28ca2-eff6-4acb-a0ba-e9c1c634d170 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:52.414494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:52.415162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:52.415444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:52.415721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:52.416458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:52.416769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:52.418149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:52.429309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 510/1000] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:52.431058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:52.497810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:52.549897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d04b0c-8389-42fb-99d0-e1f2a810e37e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:52.589052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:52.589839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:52.590221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:52.590620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:52.591513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] RBAC: Adding query filter params (interface=public, service_id=2ee2c7e249824140a5b89591b95ad9db, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:52.591926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:52.593440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabf4d4-8d59-4204-9234-d50cfa85f4f8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:52.604972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 491/1001] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:52 2022] GET /identity/v3/endpoints?service_id=2ee2c7e249824140a5b89591b95ad9db&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:52.606493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:52.716719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132b34-6c8e-4e84-91a3-bfc79a9758b5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:52.718303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:52.754152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39d8a0-4f26-454a-9167-a7c6ed4add85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:52.754832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39d8a0-4f26-454a-9167-a7c6ed4add85 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:52.755122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39d8a0-4f26-454a-9167-a7c6ed4add85 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:52.755400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39d8a0-4f26-454a-9167-a7c6ed4add85 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:52.756079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39d8a0-4f26-454a-9167-a7c6ed4add85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:52.757497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39d8a0-4f26-454a-9167-a7c6ed4add85 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:52.772345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 511/1002] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:31:52 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:31:53.510071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a6230-e7b0-4ea7-8696-f687a4f5d5b3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:53.510694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a6230-e7b0-4ea7-8696-f687a4f5d5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:53.510694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a6230-e7b0-4ea7-8696-f687a4f5d5b3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:53.512151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 492/1003] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:53.517610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:53.518076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:53.518159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:53.535660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:53.536868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:53.605546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 512/1004] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:53.632372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a6230-e7b0-4ea7-8696-f687a4f5d5b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:53.742854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a6230-e7b0-4ea7-8696-f687a4f5d5b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:53.763275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:53.779376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:53.780168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:53.780492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:53.780764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:53.781411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:53.783425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:53.795267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:53.796755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:31:53.797850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:53.800379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 493/1005] 10.222.0.16 () {56 vars in 1105 bytes} [Thu May 26 13:31:53 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:53.802829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:53.904987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:53.915930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c67d1-db40-4536-bb10-b4d91afeabba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:53.959470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:53.960142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:53.960479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:53.960760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:53.961471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:53.961781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:53.963155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a7167-d4bb-400f-be23-bda7e0a02228 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:53.973771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 513/1006] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:53.975489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:54.095936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2813c0c-7923-4df9-89e0-0bf6868c61f0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:54.131288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:54.131650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:54.131962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:54.132202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:54.132926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] RBAC: Adding query filter params (type=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:54.133232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:54.134600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c33db-76dc-4008-a1ee-25593137e8bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:54.144590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 494/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:54.250559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3d39d-1e26-4196-afb1-530d1dca11a4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:54.250917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3d39d-1e26-4196-afb1-530d1dca11a4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:54.251116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3d39d-1e26-4196-afb1-530d1dca11a4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:54.253052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 514/1008] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:54.258822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:54.259149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:54.259386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:54.277140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:54.278370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:54.330592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:54.359447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 495/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:54.382908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3d39d-1e26-4196-afb1-530d1dca11a4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:54.497017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3d39d-1e26-4196-afb1-530d1dca11a4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:54.536792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e3a2-5e1c-4938-9a8b-eee8f0ce38be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:54.537679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e3a2-5e1c-4938-9a8b-eee8f0ce38be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:54.537880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e3a2-5e1c-4938-9a8b-eee8f0ce38be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:54.538153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e3a2-5e1c-4938-9a8b-eee8f0ce38be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:54.539454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e3a2-5e1c-4938-9a8b-eee8f0ce38be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:54.540962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e3a2-5e1c-4938-9a8b-eee8f0ce38be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:54.547552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 515/1010] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:54.549162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:54.655212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb15b-468f-4556-aceb-7aea7e374d85 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:54.690165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fc308-543f-451e-9413-939a3e61c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:54.690848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fc308-543f-451e-9413-939a3e61c1f0 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:54.691131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fc308-543f-451e-9413-939a3e61c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:54.691413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fc308-543f-451e-9413-939a3e61c1f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:54.692133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fc308-543f-451e-9413-939a3e61c1f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:54.693621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fc308-543f-451e-9413-939a3e61c1f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:54.728344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 496/1011] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:54 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:55.044965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab18f1-bc54-4199-8c5d-0fe5ad949529 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:55.045306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab18f1-bc54-4199-8c5d-0fe5ad949529 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:55.045500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab18f1-bc54-4199-8c5d-0fe5ad949529 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:55.047058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 516/1012] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:55.052788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:55.053080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:55.053359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:55.070320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:55.071494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:55.096974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:55.143448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 497/1013] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:55 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:55.166175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab18f1-bc54-4199-8c5d-0fe5ad949529 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:55.202556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:55.291480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab18f1-bc54-4199-8c5d-0fe5ad949529 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:55.352253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:55.353104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:55.353390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:55.353668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:55.354337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:55.355740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:55.368262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:55.369407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:55.370685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 517/1014] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:31:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:55.377629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:55.485042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2673d04-2edf-4504-a486-fa1c42de114f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:55.520843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:55.521720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:55.522071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:55.522357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:55.523237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:55.523556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:55.525117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:55.530874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 498/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:55.537161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:55.649298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:55.660651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d404c-2f8a-43e6-a58d-788fd9ca884c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:55.711891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:55.712598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:55.712882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:55.713159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:55.713864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:55.714170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:55.715546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddcad5-7b5d-4f0f-8268-2ffc370c2b0d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:55.721838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 518/1016] 10.222.0.16 () {56 vars in 1112 bytes} [Thu May 26 13:31:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:55.727458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:55.757471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:55.840638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc0228-9326-4730-ae03-ddf157d2865e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:55.885554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee25aba-7e1c-419d-a7b6-ed6b38de24a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:55.886357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25aba-7e1c-419d-a7b6-ed6b38de24a1 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:55.886638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25aba-7e1c-419d-a7b6-ed6b38de24a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:55.886914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25aba-7e1c-419d-a7b6-ed6b38de24a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:55.887657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee25aba-7e1c-419d-a7b6-ed6b38de24a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:55.889064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee25aba-7e1c-419d-a7b6-ed6b38de24a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:55.918356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 499/1017] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:31:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:56.442931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffae7a-7256-4983-ae9e-bffb4780da15 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:56.443346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffae7a-7256-4983-ae9e-bffb4780da15 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:56.443534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffae7a-7256-4983-ae9e-bffb4780da15 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:56.444952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 519/1018] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:56.450414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cdf57-bab3-4c0e-b198-727c2e6a327c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:56.450713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cdf57-bab3-4c0e-b198-727c2e6a327c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:56.451001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cdf57-bab3-4c0e-b198-727c2e6a327c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:56.468709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8b9cdf57-bab3-4c0e-b198-727c2e6a327c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:56.470106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8b9cdf57-bab3-4c0e-b198-727c2e6a327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:56.548549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 500/1019] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:56 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:56.571146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffae7a-7256-4983-ae9e-bffb4780da15 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:56.695572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffae7a-7256-4983-ae9e-bffb4780da15 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:56.738429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0015c-7c19-4d99-a75a-4fe8f6e0afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:56.739121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0015c-7c19-4d99-a75a-4fe8f6e0afcd None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:56.739405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0015c-7c19-4d99-a75a-4fe8f6e0afcd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:56.739684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0015c-7c19-4d99-a75a-4fe8f6e0afcd None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:56.740467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0015c-7c19-4d99-a75a-4fe8f6e0afcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:56.741855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0015c-7c19-4d99-a75a-4fe8f6e0afcd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:56.764340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 520/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:31:56 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:31:57.053027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3f9a8-e86e-45b5-aaa3-d3cca1f836dc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:57.053985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3f9a8-e86e-45b5-aaa3-d3cca1f836dc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:57.054275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3f9a8-e86e-45b5-aaa3-d3cca1f836dc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:57.055677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 501/1021] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:57.061391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:57.061684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:57.061970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:57.088468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:57.089694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:57.103586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:57.178615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 521/1022] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:57.203378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3f9a8-e86e-45b5-aaa3-d3cca1f836dc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:57.213362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:57.324571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3f9a8-e86e-45b5-aaa3-d3cca1f836dc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:57.389424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6220d-0ddd-40c9-bc96-bcd0b05c2571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:57.390487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6220d-0ddd-40c9-bc96-bcd0b05c2571 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:57.391333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6220d-0ddd-40c9-bc96-bcd0b05c2571 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:57.391630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6220d-0ddd-40c9-bc96-bcd0b05c2571 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:57.393296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6220d-0ddd-40c9-bc96-bcd0b05c2571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:57.394814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6220d-0ddd-40c9-bc96-bcd0b05c2571 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:57.396791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 502/1023] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:57.403216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:57.558324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b5f1-6f2e-4322-89e7-dc74a5e70836 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:57.597627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6de8a-e04e-4105-81fc-5b5d60a37141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:57.598308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6de8a-e04e-4105-81fc-5b5d60a37141 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:57.598623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6de8a-e04e-4105-81fc-5b5d60a37141 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:57.598794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6de8a-e04e-4105-81fc-5b5d60a37141 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:57.599510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6de8a-e04e-4105-81fc-5b5d60a37141 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:57.600998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6de8a-e04e-4105-81fc-5b5d60a37141 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:57.618254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:57.628034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 522/1024] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:31:57 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:31:58.322427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405122a2-ad15-4263-b581-b0b7361f23ff None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:58.322963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405122a2-ad15-4263-b581-b0b7361f23ff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:58.322963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405122a2-ad15-4263-b581-b0b7361f23ff None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:58.324552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 503/1025] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:31:58.330234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:58.330590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:58.330858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:58.348418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:58.349634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:58.420607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 523/1026] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:31:58.444907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405122a2-ad15-4263-b581-b0b7361f23ff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:58.533834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:58.578186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405122a2-ad15-4263-b581-b0b7361f23ff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:58.623858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:58.624818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:58.625099 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:58.625906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:58.637766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:31:58.646910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:58.648026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:58.648951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 504/1027] 10.222.0.16 () {56 vars in 1120 bytes} [Thu May 26 13:31:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:58.654182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:58.768544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d2ae-a846-4d06-bc23-fce0978e3045 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:58.805226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:58.817758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:58.818464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:58.818812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:58.819307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:58.820204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:58.820577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:58.822014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:58.831783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 524/1028] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:31:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:58.837994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:58.966602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1d6c8-3180-4fcd-ab9b-4b223a7a0978 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.010693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.011511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.011792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.012067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.014395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:59.027043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.028597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5c54aa31-fb10-4d0f-8b61-58c8b62b9b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:31:59.049818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:59.050929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:59.057183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:31:59.057183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:31:59.057183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:31:59.057183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:59.057183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 505/1029] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:59.057508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.172783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cd9ea-a098-481f-a52e-31b1ea478970 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.175246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd57cc-b852-4106-9d87-b62f3c0f6453 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.175533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd57cc-b852-4106-9d87-b62f3c0f6453 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.175812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd57cc-b852-4106-9d87-b62f3c0f6453 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.177232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 506/1030] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:31:59.182679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.182971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.183252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.200636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:31:59.201832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:31:59.215294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.215977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.216258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.217432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.218178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:59.218488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.220361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.239182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 525/1031] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:59.240465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.286515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 507/1032] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:31:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:31:59.302452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:59.314234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.380352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab4c8a-02b4-4d21-8194-5590ccc37e25 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.418714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.419521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.419804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.420077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.428990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.430414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.443319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b38f3-fba5-4807-8076-5ffecd862523 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:31:59.446537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:59.447610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:31:59.449143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 526/1033] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:59.457467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.472785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:59.493804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.494771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.495061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.495448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] PATH_INFO: `/v3/services/image` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.496468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.496468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:31:59.513948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:31:59.514992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:31:59.516013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 508/1034] 10.222.0.16 () {56 vars in 1105 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:31:59.522286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.599496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad69a-2702-4c4a-a21d-b3607e997399 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.613651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:59.646438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.647137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.647435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.647710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.648626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:59.648952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.650424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.665953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b9e69-a64e-49c3-93f3-380b180e08dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.680458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 527/1035] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:31:59.686886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.715035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.715783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.716082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.716446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.717329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:59.717689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.719278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.725939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 509/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:59.731519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.786745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:31:59.847723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e06cf3-ef26-48d1-a658-f890899cad31 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.865603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510008b0-1adc-408c-a74a-39cb367aa07e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:31:59.893701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.894691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.894985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.895267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.926860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:31:59.927652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:31:59.927983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:31:59.928334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:31:59.929143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] RBAC: Adding query filter params (type=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:31:59.929486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:31:59.930990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:31:59.944340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 510/1037] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:31:59.946581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:31:59.960126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.006610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.008212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.065565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8691d7-0af4-4fe2-8e43-28812bf0a618 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.076001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.110913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.111584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.111865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.112142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.112983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] RBAC: Adding query filter params (interface=public, service_id=e0dddd14b1d347db8b64a9f52bb72b3c, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:00.113297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.114879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e6276-ce12-41c7-a8a0-d872debe275d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.120510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 511/1038] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/endpoints?service_id=e0dddd14b1d347db8b64a9f52bb72b3c&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:00.126986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512291e1-c6a5-4de0-a476-aeacd37965e6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.127266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512291e1-c6a5-4de0-a476-aeacd37965e6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.127549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512291e1-c6a5-4de0-a476-aeacd37965e6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.129231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 512/1039] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:00.129973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512291e1-c6a5-4de0-a476-aeacd37965e6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.131502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 528/1040] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:31:59 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:32:00.137216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.226323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.270687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b0c30-0805-4899-a51c-cef3fad51c5c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.275160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512291e1-c6a5-4de0-a476-aeacd37965e6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.319396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.320112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.320422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.320701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.325661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf7d50-e3da-4cc1-801b-913942ddc2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.326339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf7d50-e3da-4cc1-801b-913942ddc2bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.326626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf7d50-e3da-4cc1-801b-913942ddc2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.326906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf7d50-e3da-4cc1-801b-913942ddc2bf None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.327597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf7d50-e3da-4cc1-801b-913942ddc2bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.330786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf7d50-e3da-4cc1-801b-913942ddc2bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.332988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.334545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.338360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 513/1041] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/services => generated 1957 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:32:00.339681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.339974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.340253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.363400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:00.364689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:00.368555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:00.369597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:00.370579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:32:00.370579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:00.370579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:00.370579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:00.379944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 529/1042] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:00.382927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.415388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.486006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 514/1043] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:00.510658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.544699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fc841-2b5b-46f0-8be2-9158ecea59f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.571639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.600490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.601190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.601475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.601758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.602511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:00.602819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.604245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.617831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 530/1044] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:00.623884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.663804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e44a2-ffb9-4299-b175-4af82069b47f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.716546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.717938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.718441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.719521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.732172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:00.740617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:00.744748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:00.746385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 515/1045] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:00.749487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.771703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84290b-f7ab-4f29-878b-2ec23fedfe9a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.784838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.833044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.833934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.834132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.834406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.842661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.844737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:32:00.860733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:00.867308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:00.868363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 531/1046] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:00.870464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:00.876513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98604e-50e7-413c-8633-00fff4cb11cb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:00.924943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:00.926410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:00.926410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:00.926668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:00.927268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:00.927583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:00.929048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:00.938940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:00.945334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 516/1047] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:00.947387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:01.018047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e6806-ae2f-4346-95ba-6cc7e334908e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:01.060342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:01.069156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:01.069846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:01.070177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:01.070459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:01.071278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:01.071609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:01.073202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:01.087740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de18b6-2499-4cd7-a9a3-56de6cc9cb1c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:01.093737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 532/1048] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:01.095306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:01.147940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:01.148894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:01.149173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:01.149450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] PATH_INFO: `/v3/users/0dc8e4d2907440aea7f2037b817dd715` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:01.169506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] RBAC: Authorizing `identity:get_user(user_id=0dc8e4d2907440aea7f2037b817dd715)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:01.173425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:01.183684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 517/1049] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:00 2022] GET /identity/v3/users/0dc8e4d2907440aea7f2037b817dd715 => generated 285 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:01.185557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:01.243491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:01.247121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd0091-1b4b-469e-bbd8-e30936f015d4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:01.291978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:01.292677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:01.292967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:01.293254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:01.294113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] RBAC: Adding query filter params (role.id=414f73f64d774dec8cafd23a785fd4c7, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=535f23db24704422a4128ff09f1cc284) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:01.294438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:01.295954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:01.312344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 533/1050] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:01 2022] GET /identity/v3/role_assignments?user.id=535f23db24704422a4128ff09f1cc284&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=414f73f64d774dec8cafd23a785fd4c7 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:01.342857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838fcc-e664-4868-8efe-530a09af50ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:01.395872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:01.396576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:01.396877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:01.397151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:01.421749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:01.504827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:01.507212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:01.645602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 518/1051] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:32:01.651344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:01.778617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b402b3f-76cd-4d30-8e26-b1cd4bb985fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:01.818766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89989047-cbf5-4406-b057-01bfb89f7166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:01.819645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89989047-cbf5-4406-b057-01bfb89f7166 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:01.819935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89989047-cbf5-4406-b057-01bfb89f7166 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:01.820214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89989047-cbf5-4406-b057-01bfb89f7166 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:01.821505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89989047-cbf5-4406-b057-01bfb89f7166 None admin] RBAC: Authorizing `identity:get_project(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:01.822979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89989047-cbf5-4406-b057-01bfb89f7166 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:01.825131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 534/1052] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:01 2022] GET /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:01.831760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:01.887044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:01.970191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9968-110a-4aea-af1e-9c9b78d3c4d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:02.023680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:02.024370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:02.024648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:02.024920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:02.025757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e, user.id=0dc8e4d2907440aea7f2037b817dd715) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:02.026064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:02.027536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fdfbc-35a5-4835-adb4-531696996455 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:02.034794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 519/1053] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:01 2022] GET /identity/v3/role_assignments?user.id=0dc8e4d2907440aea7f2037b817dd715&scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:02.777565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9a15a-5e2d-41ea-af5c-abba5ba9a326 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:02.777873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9a15a-5e2d-41ea-af5c-abba5ba9a326 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:02.778205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9a15a-5e2d-41ea-af5c-abba5ba9a326 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:02.779525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 535/1054] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:02.786504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:02.787140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:02.787140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:02.807939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:02.809625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:02.945021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 520/1055] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:02.986498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9a15a-5e2d-41ea-af5c-abba5ba9a326 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:03.044228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:03.130825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9a15a-5e2d-41ea-af5c-abba5ba9a326 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:03.169817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:03.170620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:03.171305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:03.171582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:03.172466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:03.173789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:03.191793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:03.193401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 13:32:03.195365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:03.197108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 536/1056] 10.222.0.16 () {56 vars in 1105 bytes} [Thu May 26 13:32:02 2022] GET /identity/v3/services/image => generated 86 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:03.199376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:03.247954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:03.323858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61d088-e011-4be9-9878-a6e3819db2ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:03.366893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:03.367611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:03.367889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:03.368167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:03.368981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:03.369293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:03.370795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:03.377735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 521/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:03.384157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:03.556816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:03.568076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cbad9-df07-495f-99bc-9663ef9ca177 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:03.619714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:03.620350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:03.620634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:03.620919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:03.621657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:03.621969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:03.623397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c35bb-7bb2-411f-a5f2-f1a79c700ea7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:03.630201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 537/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:03.636846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:03.744473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dee1d2-960f-4309-82b5-02d8455403ee None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:03.745043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dee1d2-960f-4309-82b5-02d8455403ee None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:03.745454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dee1d2-960f-4309-82b5-02d8455403ee None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:03.746880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 538/1059] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:03.759204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:03.759606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:03.759790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:03.777857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:03.778619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d26038-2e83-445b-a35a-b69b68eafc23 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:03.779066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:03.783292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:03.826141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:03.826826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:03.827103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:03.827384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:03.828226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:03.829708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:03.854031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 522/1060] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:32:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:32:03.904431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 539/1061] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:32:03.942668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.096816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05b7a9-8b40-4371-af71-672d725f330e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:04.139599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:04.140449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.141311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.141676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.157355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:04.165633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:04.166672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:04.167950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 523/1062] 10.222.0.16 () {56 vars in 1120 bytes} [Thu May 26 13:32:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:04.173843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.280488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:04.300157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a425c5-b5dd-4279-aeb2-317f7f3f87a7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:04.343232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:04.344077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.344447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.344684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.345597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:04.345798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:04.347373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cabf90-c898-4d4c-9e06-a83cfa5bb820 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:04.363003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 540/1063] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:32:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:04.364524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.393334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:04.487537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d93c0-7d34-45fa-9f32-d66da1ba5895 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:04.544676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:04.545609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.545893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.546174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.554069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e1968-2088-4d3c-a519-1490da9bb886 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.554666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e1968-2088-4d3c-a519-1490da9bb886 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.555171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e1968-2088-4d3c-a519-1490da9bb886 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.557489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 541/1064] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:04.563630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:04.565936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.566235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.566355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:04.566523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.585264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:04.586598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:04.587692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:04.588585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:04.590416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:32:04.590416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:04.590416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:04.590416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:04.590416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 524/1065] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:04.594850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.674019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 542/1066] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:32:04.696928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.697159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:04.732089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ddd87-9951-41af-b107-2b1571628cde None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:04.775936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:04.776694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.777000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.777282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.778015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:04.778321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:04.779738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:04.791902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:04.798682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 525/1067] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:04.799897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.826643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390b68f-390d-4a0a-9ad8-0e92dc968e82 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:04.878513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:04.879970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.880532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.881021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:04.899268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:04.914074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:04.916941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:04.918737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:04.924339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 543/1068] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:04.925280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:04.946140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f360c-e09a-48db-bc74-44dca6c0e238 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:04.985592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:04.991760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:04.992612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:04.992928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:04.993162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.001911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.003397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:32:05.020535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:05.022085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:05.023165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 526/1069] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:32:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:05.028154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.068117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269cf16-dd9a-4a23-8eae-dbbf7e32cb25 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.117616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.118788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.118788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.119003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.119711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:05.120018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.121455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.129776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:05.136644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 544/1070] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:05.140858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.160292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0598a8-c366-4f89-aefd-39630dd4d61a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.214529 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.215698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.216211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.216799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.218196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:05.219330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.223014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.252951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 527/1071] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:32:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:05.256374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.283519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f03cf-f40b-4309-8343-1999433dc48f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.333291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.334801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.335075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:05.335334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.335985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] PATH_INFO: `/v3/users/0dc8e4d2907440aea7f2037b817dd715` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.337633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] RBAC: Authorizing `identity:get_user(user_id=0dc8e4d2907440aea7f2037b817dd715)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.339707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.347588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 545/1072] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:05 2022] GET /identity/v3/users/0dc8e4d2907440aea7f2037b817dd715 => generated 285 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:05.349321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.417478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc9075-39fe-48d0-a91c-1ae8badb68dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.449746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:05.463943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.464776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.465312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.466294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.486121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.488442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.506315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac21b3-89d4-483c-bbfd-af7835829b92 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:05.513454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:05.514880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:05.516228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:32:05.516228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:05.516228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:05.516228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:05.517094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 528/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:05.523722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.550888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.552043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.552360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.552624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.553894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] RBAC: Authorizing `identity:get_project(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.555355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.557319 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 546/1074] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:05 2022] GET /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e => generated 311 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:05.564572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.669887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db18c4-aaec-493b-8f6e-98625fef08e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.671492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:05.717312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.718389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.718894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.719274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655f7e0-e529-41e7-8e43-980c618e6125 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.719511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.722327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:05.722327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.726774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.744719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 529/1075] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:05.752144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:05.777241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.778219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.778508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.778811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e/users/0dc8e4d2907440aea7f2037b817dd715/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.780973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] RBAC: Authorizing `identity:create_grant(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e, user_id=0dc8e4d2907440aea7f2037b817dd715, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.782932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:05.786912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:05.812523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 547/1076] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:32:05 2022] PUT /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e/users/0dc8e4d2907440aea7f2037b817dd715/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:32:05.890349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7e039-bc04-45f4-9ff5-2d70ac994120 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:05.964514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:05.965946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:05.966455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:05.966964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:05.981724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:05.984928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:32:06.021920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:06.022960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:32:06.023889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:06.024766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 530/1077] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:32:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:06.030691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:06.180876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbcec7-405c-4c4a-9d12-d806afa4305a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:06.224435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:06.225129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:06.225407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:06.225685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:06.226501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:06.226842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:06.228350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f77f2-27f3-4f07-b9a6-9994278328b5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:06.242466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 548/1078] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:06.248788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:06.283419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:06.283768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825680cb-3626-4683-a3ed-c5abbc97cec5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:06.284291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825680cb-3626-4683-a3ed-c5abbc97cec5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:06.284849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825680cb-3626-4683-a3ed-c5abbc97cec5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:06.288427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 549/1079] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:32:06.298467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:06.299176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:06.299690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:06.330053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:06.331542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:06.418537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:06.420082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1b69-71d4-4588-8072-4dd6f2a2b2b6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:06.440022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 550/1080] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:32:06.471633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:06.472854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:06.473311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:06.473742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/535f23db24704422a4128ff09f1cc284/roles/414f73f64d774dec8cafd23a785fd4c7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:06.476333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=535f23db24704422a4128ff09f1cc284, role_id=414f73f64d774dec8cafd23a785fd4c7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:06.478059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:06.480073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:06.502710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 531/1081] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:32:06 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/535f23db24704422a4128ff09f1cc284/roles/414f73f64d774dec8cafd23a785fd4c7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:32:06.654858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c187c70-effb-4109-a6e1-5a26a9ec37f5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:06.725303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:06.725914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:06.726232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:06.726507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:06.791876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:06.809034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:06.810509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:06.952194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 551/1082] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:32:06.955679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:07.089583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:07.127932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30247-88ec-43c6-a23c-fd6a3c9eecfd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:07.174677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:07.175436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:07.175719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:07.175986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:07.187732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:07.189231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7b7fcf08-e53e-4533-b6a7-116596bc6863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:07.210307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:07.211455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:07.212925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:32:07.212925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:07.212925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:07.212925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:07.212925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 532/1083] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:07.217632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:07.320821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:07.346520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece329d-f834-4d00-82f2-90402da0c499 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:07.384499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:07.385181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:07.385470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:07.385753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:07.386493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:07.386806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:07.388208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a119fe1-1472-4b95-938c-eb9d062ec2ae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:07.405379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 552/1084] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:08.211834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003f144-0e44-44f7-8cbf-3a9887495cca None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.212392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003f144-0e44-44f7-8cbf-3a9887495cca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.212483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003f144-0e44-44f7-8cbf-3a9887495cca None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.213837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 533/1085] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:08.220071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.220701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.220815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.237777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:08.238988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:08.313266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 553/1086] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:08.338125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003f144-0e44-44f7-8cbf-3a9887495cca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:08.462125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003f144-0e44-44f7-8cbf-3a9887495cca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:08.506334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:08.507154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.507442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.507717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.509955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:08.520782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:08.528773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:08.530055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:08.531454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 534/1087] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:08.536934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:08.623313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:08.650772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b8d07-53f4-4b8a-afac-bc9cbd59499f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:08.688499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:08.689181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.689458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.689730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.690485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:08.690792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:08.692225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:08.701964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 554/1088] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:08.709619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:08.836908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743de21-406b-4509-b245-cbe8fda71a46 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:08.873880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7dd8a4-a816-4cf0-b9ea-6083b4226c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:08.875389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dd8a4-a816-4cf0-b9ea-6083b4226c6e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.875708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dd8a4-a816-4cf0-b9ea-6083b4226c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.875985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dd8a4-a816-4cf0-b9ea-6083b4226c6e None admin] PATH_INFO: `/v3/users/0dc8e4d2907440aea7f2037b817dd715` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.877250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7dd8a4-a816-4cf0-b9ea-6083b4226c6e None admin] RBAC: Authorizing `identity:get_user(user_id=0dc8e4d2907440aea7f2037b817dd715)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:08.878714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7dd8a4-a816-4cf0-b9ea-6083b4226c6e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:08.880382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 535/1089] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:08 2022] GET /identity/v3/users/0dc8e4d2907440aea7f2037b817dd715 => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:08.886627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:08.945501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335795ab-d5ae-4370-8680-cf8c9d66d6ee None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.946048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335795ab-d5ae-4370-8680-cf8c9d66d6ee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.946565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335795ab-d5ae-4370-8680-cf8c9d66d6ee None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.949207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 536/1090] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:32:08.958844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:08.959340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:08.959848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:08.989459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:08.991576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:09.032997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413679e-dbaf-46a1-a56b-f23dd38fe5a8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.078871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:09.088677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.089557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.090044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.090499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.097360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 537/1091] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:32:09.102036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:09.122363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:09.185971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:09.188092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:09.248707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdfc21-614a-4646-8b4e-556d1708edb7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.290570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.291415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.291695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.291970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.304425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:09.313496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:09.314532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:09.315388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 538/1092] 10.222.0.16 () {56 vars in 1120 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:09.323139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:09.346608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 555/1093] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:09.356346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:09.392748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:09.410267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:09.462021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd254bea-bf13-484f-81aa-2fcfb380b57a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.506318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa77017-6954-40c7-964f-cb394e42ced0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.509934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.512856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.512856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.512856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.513220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:09.513814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:09.519858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:09.528011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 539/1094] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:09.536777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:09.563503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.564520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.564803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.565081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.566474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] RBAC: Authorizing `identity:get_project(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:09.568111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:09.569891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 556/1095] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:09.577278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:09.665589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e736d11-4966-4937-9dc2-3ab719ca6105 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.685319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:09.710196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.711004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.711287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.711562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.711654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b6c28-c5e5-49ef-8e9a-edeb72482747 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.723864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:09.725430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:09.727118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:09.747108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:09.748174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:09.750401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:32:09.750401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:09.750401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:09.750401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:09.750401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 540/1096] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:32:09.755045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:09.761037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.762027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.762518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.763025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.764556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e, user.id=0dc8e4d2907440aea7f2037b817dd715) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:09.764986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:09.767657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:09.775140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 557/1097] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/role_assignments?user.id=0dc8e4d2907440aea7f2037b817dd715&scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:09.884560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4737643-21b9-4b45-985b-6ab98a285fdd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:09.933235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:09.933718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:09.934003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:09.934282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:09.935028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:09.935338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:09.936802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:09.952651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 541/1098] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:09.960232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:10.092630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dceca5-b294-4e7f-8678-679a4147a751 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:10.128945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:10.129760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:10.130045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:10.130327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:10.138670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:10.140145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:32:10.155290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:10.157105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:10.158151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 558/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Thu May 26 13:32:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:10.163340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:10.191216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:10.272710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3d6f4-3796-4315-98ad-e6e87f340e2f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:10.308932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:10.309611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:10.310472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:10.310472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:10.311076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:10.311391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:10.312911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:10.333564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 542/1100] 10.222.0.16 () {56 vars in 1133 bytes} [Thu May 26 13:32:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:10.340109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:10.382390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:10.456236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c014e7e-aba1-4022-b25d-18b80c7630a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:10.491657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:10.492336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:10.492629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:10.492911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:10.566805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:10.568276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:10.600503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:10.674946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 559/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:10.681011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:10.778687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:10.795426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061091d1-d96b-4e64-8802-ed029c6ba6b1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:10.831176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:10.832053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:10.832362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:10.832905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:10.843768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:10.845448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:10.863189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:10.864275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:10.865269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:32:10.865269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:10.865269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:32:10.865269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:10.868464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 543/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:10.870079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:10.971760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86c09-97c0-4b86-90b4-7762034a2679 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:11.003570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:11.010446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:11.011136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:11.011447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:11.011753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:11.012580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:11.012927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:11.014379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:11.025996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 560/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:11.031669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:11.132373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dcd93-e04e-45b9-acfb-0a7f834d8825 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:11.174107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:11.174896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:11.175176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:11.175451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:11.178123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:11.182717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:11.184472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:32:11.199255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:11.200225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:11.201182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 544/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:32:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:11.206514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:11.311220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58232-2049-431d-b4f9-6657c5412f98 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:11.347513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:11.348185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:11.348504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:11.348818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:11.349612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:11.349922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:11.351392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3c979-3713-4420-a67a-b602777b8053 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:11.365998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 561/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:11.372094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:11.477598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6d514-3cee-42e5-a67d-17302a1b1bdb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:11.515560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:11.516216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:11.516527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:11.516809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:11.517634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] RBAC: Adding query filter params (role.id=414f73f64d774dec8cafd23a785fd4c7, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=535f23db24704422a4128ff09f1cc284) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:11.517947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:11.519393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b1b7d-5fda-4364-aea3-c7266f2d8f82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:11.525528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 545/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:11 2022] GET /identity/v3/role_assignments?user.id=535f23db24704422a4128ff09f1cc284&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=414f73f64d774dec8cafd23a785fd4c7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:12.113266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13ab5-23ff-4bf0-bd37-cc7d764518dc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:12.113782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13ab5-23ff-4bf0-bd37-cc7d764518dc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:12.113876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13ab5-23ff-4bf0-bd37-cc7d764518dc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:12.115201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 562/1107] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:12.120904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:12.121200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:12.121488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:12.138876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:12.140087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:12.197638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:12.211000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 546/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:12.237247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13ab5-23ff-4bf0-bd37-cc7d764518dc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:12.290886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:12.344667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13ab5-23ff-4bf0-bd37-cc7d764518dc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:12.380891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d1b4c-6024-4078-ac86-cee06e10dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:12.381581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1b4c-6024-4078-ac86-cee06e10dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:12.381861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1b4c-6024-4078-ac86-cee06e10dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:12.382132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1b4c-6024-4078-ac86-cee06e10dcc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:12.383301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d1b4c-6024-4078-ac86-cee06e10dcc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:12.384763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d1b4c-6024-4078-ac86-cee06e10dcc1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:12.386379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 563/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:12.392901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:12.500941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31650-cf51-44f9-a75f-a9d5fdbfd5cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:12.535727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759feb3b-79ea-41ad-b8fc-99b16bcc16be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:12.536446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759feb3b-79ea-41ad-b8fc-99b16bcc16be None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:12.536759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759feb3b-79ea-41ad-b8fc-99b16bcc16be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:12.537058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759feb3b-79ea-41ad-b8fc-99b16bcc16be None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:12.537779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759feb3b-79ea-41ad-b8fc-99b16bcc16be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:12.539226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759feb3b-79ea-41ad-b8fc-99b16bcc16be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:12.561110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 547/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:32:12 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:32:14.824612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8280d0-f7b6-4c7a-bfd9-0bcfc2c67473 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:14.825151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8280d0-f7b6-4c7a-bfd9-0bcfc2c67473 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:14.825151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8280d0-f7b6-4c7a-bfd9-0bcfc2c67473 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:14.826526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 564/1111] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:14.832969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:14.833461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:14.833583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:14.850986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:14.852220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:14.896242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:14.924121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 548/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:14.947527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8280d0-f7b6-4c7a-bfd9-0bcfc2c67473 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:15.051414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8280d0-f7b6-4c7a-bfd9-0bcfc2c67473 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:15.068662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:15.086633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:15.087428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:15.087712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:15.087991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] PATH_INFO: `/v3/roles/9508980101c44da48df022abc45e1e60` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:15.089485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] RBAC: Authorizing `identity:get_role(role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:15.090861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:15.092295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 565/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:14 2022] GET /identity/v3/roles/9508980101c44da48df022abc45e1e60 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:15.098216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:15.197835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a5fae-340f-4c94-afe6-d10d94d293e7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:15.232975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:15.233985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:15.234238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:15.234517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] PATH_INFO: `/v3/users/e48d3aca2d774fbfa44761811d09ada9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:15.251331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] RBAC: Authorizing `identity:get_user(user_id=e48d3aca2d774fbfa44761811d09ada9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:15.252994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:15.254712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 549/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:15 2022] GET /identity/v3/users/e48d3aca2d774fbfa44761811d09ada9 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:15.260575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:15.358157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de242-f064-4675-ae2d-edca9ba84fd7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:15.392416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:15.393057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:15.393342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:15.393619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:15.446305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:15.460068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:15.461553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:15.566425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 566/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:15.572375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:15.606281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:15.683167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16c67-3a2b-4146-bcf7-a93d3a7ee190 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:15.719625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f49538-ec32-444e-9f00-7787566784c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:15.720522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f49538-ec32-444e-9f00-7787566784c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:15.720809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f49538-ec32-444e-9f00-7787566784c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:15.721087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f49538-ec32-444e-9f00-7787566784c0 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:15.722295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f49538-ec32-444e-9f00-7787566784c0 None admin] RBAC: Authorizing `identity:get_project(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:15.723748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f49538-ec32-444e-9f00-7787566784c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:15.725493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 550/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:15 2022] GET /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:15.731773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:15.824396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:15.840466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7f004-738d-47aa-84b6-9586ff859e1d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:15.876992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:15.877653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:15.877940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:15.878216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:15.879046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] RBAC: Adding query filter params (role.id=9508980101c44da48df022abc45e1e60, scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e, user.id=e48d3aca2d774fbfa44761811d09ada9) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:15.879353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:15.880846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd70d9-307e-444f-bc98-3af2f456bb39 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:15.886735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 567/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:15 2022] GET /identity/v3/role_assignments?user.id=e48d3aca2d774fbfa44761811d09ada9&scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e&role.id=9508980101c44da48df022abc45e1e60 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:18.063727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fd72c-0813-4f8a-b6bf-1dc3f4fb4c38 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:18.064273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fd72c-0813-4f8a-b6bf-1dc3f4fb4c38 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:18.064273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fd72c-0813-4f8a-b6bf-1dc3f4fb4c38 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:18.065667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 551/1118] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:18.071113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:18.071525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:18.071801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:18.089886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:18.091097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:18.157769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 568/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:18.180107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fd72c-0813-4f8a-b6bf-1dc3f4fb4c38 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:18.285132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fd72c-0813-4f8a-b6bf-1dc3f4fb4c38 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:18.286686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:18.322923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:18.323784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:18.324063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:18.324374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] PATH_INFO: `/v3/roles/9508980101c44da48df022abc45e1e60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:18.325883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] RBAC: Authorizing `identity:get_role(role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:18.327287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:18.329318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 552/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:18 2022] GET /identity/v3/roles/9508980101c44da48df022abc45e1e60 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:18.335448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:18.418220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:18.439614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c0a9-357a-4e46-801f-98e82c597c6d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:18.473927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:18.474810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:18.475105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:18.475385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] PATH_INFO: `/v3/users/e48d3aca2d774fbfa44761811d09ada9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:18.476639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] RBAC: Authorizing `identity:get_user(user_id=e48d3aca2d774fbfa44761811d09ada9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:18.478109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:18.479773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 569/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:18 2022] GET /identity/v3/users/e48d3aca2d774fbfa44761811d09ada9 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:18.486245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:18.587575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65207131-f96c-498f-ade1-03d8f506d97f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:18.623173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59c979-70b1-48a0-988e-8e87ed848c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:18.624035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59c979-70b1-48a0-988e-8e87ed848c88 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:18.625047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59c979-70b1-48a0-988e-8e87ed848c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:18.625329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59c979-70b1-48a0-988e-8e87ed848c88 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:18.626520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59c979-70b1-48a0-988e-8e87ed848c88 None admin] RBAC: Authorizing `identity:get_project(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:18.627966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59c979-70b1-48a0-988e-8e87ed848c88 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:18.629856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 553/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:18 2022] GET /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:18.635859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:18.735975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87915fb2-6ecd-4a4f-975c-5d238bf809fd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:18.769794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526639e-5fc3-447c-998a-c67781af3102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:18.770684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526639e-5fc3-447c-998a-c67781af3102 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:18.770970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526639e-5fc3-447c-998a-c67781af3102 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:18.771255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526639e-5fc3-447c-998a-c67781af3102 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e/users/e48d3aca2d774fbfa44761811d09ada9/roles/9508980101c44da48df022abc45e1e60` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:18.773502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526639e-5fc3-447c-998a-c67781af3102 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e, user_id=e48d3aca2d774fbfa44761811d09ada9, role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:18.774956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526639e-5fc3-447c-998a-c67781af3102 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:18.788737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 570/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:32:18 2022] PUT /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e/users/e48d3aca2d774fbfa44761811d09ada9/roles/9508980101c44da48df022abc45e1e60 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:32:21.044192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3740f2-b674-4d4b-84f0-ebc336c33d94 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:21.044796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3740f2-b674-4d4b-84f0-ebc336c33d94 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:21.044796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3740f2-b674-4d4b-84f0-ebc336c33d94 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:21.046777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 554/1124] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:21.052724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:21.053427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:21.053600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:21.071109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:21.072355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:21.074657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:21.143607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 571/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:21.168091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3740f2-b674-4d4b-84f0-ebc336c33d94 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:21.212383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:21.273345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3740f2-b674-4d4b-84f0-ebc336c33d94 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:21.308638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:21.309514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:21.309796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:21.310074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] PATH_INFO: `/v3/roles/9508980101c44da48df022abc45e1e60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:21.311638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] RBAC: Authorizing `identity:get_role(role_id=9508980101c44da48df022abc45e1e60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:21.313152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:21.314555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 555/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:21 2022] GET /identity/v3/roles/9508980101c44da48df022abc45e1e60 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:21.320506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:21.421034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5ca7b-dd4a-4b8f-8fcf-14d0beba7ddf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:21.456479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:21.457616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:21.457904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:21.458176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] PATH_INFO: `/v3/users/e48d3aca2d774fbfa44761811d09ada9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:21.459443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] RBAC: Authorizing `identity:get_user(user_id=e48d3aca2d774fbfa44761811d09ada9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:21.460931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:21.462602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 572/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:21 2022] GET /identity/v3/users/e48d3aca2d774fbfa44761811d09ada9 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:21.469761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:21.574770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef882fc-70ec-4c57-8a0c-c2f792b78e4f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:21.583442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:21.616267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:21.616977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:21.617261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:21.617904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:21.688855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:21.690346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:21.791045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 556/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:21.797735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:21.806832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:21.907440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f2262-87d6-47b2-8164-21b8e8e5f672 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:21.943929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfa8f-9778-4501-9e19-a373d6acafc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:21.944750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfa8f-9778-4501-9e19-a373d6acafc3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:21.945037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfa8f-9778-4501-9e19-a373d6acafc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:21.945317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfa8f-9778-4501-9e19-a373d6acafc3 None admin] PATH_INFO: `/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:21.946502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cfa8f-9778-4501-9e19-a373d6acafc3 None admin] RBAC: Authorizing `identity:get_project(project_id=aa382b8bd86e4478ae2ef8c1b5d1922e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:21.948006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cfa8f-9778-4501-9e19-a373d6acafc3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:21.949943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 573/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:21 2022] GET /identity/v3/projects/aa382b8bd86e4478ae2ef8c1b5d1922e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:21.956249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:21.965547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:22.060647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bfd11-34e8-4204-bceb-66c96af4dcb3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:22.103785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:22.104479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:22.104766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:22.105047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:22.105874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] RBAC: Adding query filter params (role.id=9508980101c44da48df022abc45e1e60, scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e, user.id=e48d3aca2d774fbfa44761811d09ada9) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:22.106190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:22.107641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a498-551b-44d2-aa53-6f853e09ce41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:22.114142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 557/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:21 2022] GET /identity/v3/role_assignments?user.id=e48d3aca2d774fbfa44761811d09ada9&scope.project.id=aa382b8bd86e4478ae2ef8c1b5d1922e&role.id=9508980101c44da48df022abc45e1e60 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:24.417443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b7f5-b58a-472d-ba0b-088eb2677433 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:24.418088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b7f5-b58a-472d-ba0b-088eb2677433 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:24.418088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b7f5-b58a-472d-ba0b-088eb2677433 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:24.419371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 574/1131] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:24.425084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:24.425496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:24.425623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:24.442420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:24.443588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:24.504691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:24.509618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 558/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:24.533164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b7f5-b58a-472d-ba0b-088eb2677433 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:24.611536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:24.637596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b7f5-b58a-472d-ba0b-088eb2677433 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:24.672032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773145ea-4637-4d8c-9765-c4b7abb2d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:24.672787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773145ea-4637-4d8c-9765-c4b7abb2d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:24.673072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773145ea-4637-4d8c-9765-c4b7abb2d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:24.673352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773145ea-4637-4d8c-9765-c4b7abb2d1d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:24.674516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773145ea-4637-4d8c-9765-c4b7abb2d1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:24.675975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773145ea-4637-4d8c-9765-c4b7abb2d1d8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:24.677633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 575/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:24.683586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:24.781386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd2a38-0d1a-4845-a07b-a38c1d450d4c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:24.814575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea200b4b-93af-43e8-a1de-e2e03424d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:24.815260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea200b4b-93af-43e8-a1de-e2e03424d6a0 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:24.815547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea200b4b-93af-43e8-a1de-e2e03424d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:24.815833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea200b4b-93af-43e8-a1de-e2e03424d6a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:24.816568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea200b4b-93af-43e8-a1de-e2e03424d6a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:24.818016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea200b4b-93af-43e8-a1de-e2e03424d6a0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:24.843780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 559/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:32:24 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:32:27.150969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f389aaf-5db1-4a17-9c7a-da6d3b276495 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:27.151513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f389aaf-5db1-4a17-9c7a-da6d3b276495 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:27.151513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f389aaf-5db1-4a17-9c7a-da6d3b276495 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:27.152916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 576/1135] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:27.158798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:27.159257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:27.159352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:27.176370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:27.177485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:27.222475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:27.243502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 560/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:32:27.273133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f389aaf-5db1-4a17-9c7a-da6d3b276495 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:27.381595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f389aaf-5db1-4a17-9c7a-da6d3b276495 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:27.415450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec785a0-049e-4722-81df-638f60165f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:27.416238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec785a0-049e-4722-81df-638f60165f63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:27.416549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec785a0-049e-4722-81df-638f60165f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:27.416825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec785a0-049e-4722-81df-638f60165f63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:27.417992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec785a0-049e-4722-81df-638f60165f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:27.419429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec785a0-049e-4722-81df-638f60165f63 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:27.421120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 577/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:32:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:27.427064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:27.526527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bc8c0-cbab-43c4-9963-4b0c9ce52234 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:27.570571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a733b7-197d-4a18-8ec9-8441094a588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:27.571241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a733b7-197d-4a18-8ec9-8441094a588d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:27.571524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a733b7-197d-4a18-8ec9-8441094a588d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:27.571801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a733b7-197d-4a18-8ec9-8441094a588d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:27.572541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a733b7-197d-4a18-8ec9-8441094a588d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:27.573997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a733b7-197d-4a18-8ec9-8441094a588d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:27.595887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 561/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:32:27 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:32:29.995388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f25e-e409-49dd-892e-effe62fe68dd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:29.995937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f25e-e409-49dd-892e-effe62fe68dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:29.996049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f25e-e409-49dd-892e-effe62fe68dd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:29.997373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 578/1139] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:30.003271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:30.003672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:30.003828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:30.020813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:30.021980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:30.050581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:30.091242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 562/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:30.114499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f25e-e409-49dd-892e-effe62fe68dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:30.127493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:30.215426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f25e-e409-49dd-892e-effe62fe68dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:30.247948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:30.249284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:30.249567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:30.249960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:30.260543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:30.268738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:30.269973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:30.275988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 579/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:30.276982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:30.372716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989468d4-060b-4a27-a4f0-24147ac35122 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:30.405785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:30.406497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:30.406782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:30.407061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:30.407803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:30.408109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:30.409512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:30.417405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 563/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:30.423794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:30.494126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:30.525189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563f2dd-0bef-4da8-a261-df200487b05b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:30.559511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:30.560393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:30.560677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:30.560946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] PATH_INFO: `/v3/users/5a13bd21079d486db22383667db1d531` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:30.574467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] RBAC: Authorizing `identity:get_user(user_id=5a13bd21079d486db22383667db1d531)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:30.575899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:30.577783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 580/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:30 2022] GET /identity/v3/users/5a13bd21079d486db22383667db1d531 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:30.583759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:30.614022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:30.682223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c11c9-f41d-48da-9371-69eb4649d018 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:30.715039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:30.715674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:30.715955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:30.716231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:30.784553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:30.785945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:30.815229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:30.885403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 564/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:30.891403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:30.992947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4fab8-e9a6-4288-b93a-1c9f4931947e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:31.021718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:31.029922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134ab86-931a-465b-9372-0c1966f9a8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:31.030840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ab86-931a-465b-9372-0c1966f9a8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:31.031194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ab86-931a-465b-9372-0c1966f9a8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:31.031479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134ab86-931a-465b-9372-0c1966f9a8a6 None admin] PATH_INFO: `/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:31.032693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ab86-931a-465b-9372-0c1966f9a8a6 None admin] RBAC: Authorizing `identity:get_project(project_id=40507a9aaa6a4fb681b9a91b92f9d369)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:31.034143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134ab86-931a-465b-9372-0c1966f9a8a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:31.036939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 581/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:30 2022] GET /identity/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:31.042369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:31.144665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1590d-ccfc-4985-9c08-c8d511339cdb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:31.173310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:31.179051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:31.179704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:31.179992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:31.180276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:31.181133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=40507a9aaa6a4fb681b9a91b92f9d369, user.id=5a13bd21079d486db22383667db1d531) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:31.181450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:31.182896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68401c-aeef-435e-9fc7-b3c7d7b11405 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:31.188940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 565/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:31 2022] GET /identity/v3/role_assignments?user.id=5a13bd21079d486db22383667db1d531&scope.project.id=40507a9aaa6a4fb681b9a91b92f9d369&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:33.362609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7b55e-d9c2-4cb9-8009-c8d7bbec83ec None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:33.363171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7b55e-d9c2-4cb9-8009-c8d7bbec83ec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:33.363171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7b55e-d9c2-4cb9-8009-c8d7bbec83ec None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:33.364548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 582/1147] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:33.370128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:33.370566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:33.370686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:33.388756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:33.390137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:33.462286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 566/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:33.485305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7b55e-d9c2-4cb9-8009-c8d7bbec83ec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:33.585475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7b55e-d9c2-4cb9-8009-c8d7bbec83ec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:33.618764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:33.619476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:33.619756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:33.620029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:33.630512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:33.638012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:33.639225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:33.640110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 583/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:33.645373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:33.742224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9def69-0173-465f-9555-b932f3135df4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:33.765355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:33.776195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:33.776879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:33.777161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:33.777437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:33.778171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:33.778474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:33.780568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:33.788682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 567/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:33.794805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:33.850469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:33.895968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca301d81-119d-4f12-bb55-f391a34e1f7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:33.928869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:33.929740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:33.930030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:33.930305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] PATH_INFO: `/v3/users/5a13bd21079d486db22383667db1d531` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:33.931483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] RBAC: Authorizing `identity:get_user(user_id=5a13bd21079d486db22383667db1d531)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:33.932967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:33.934592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 584/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:33 2022] GET /identity/v3/users/5a13bd21079d486db22383667db1d531 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:33.941093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:34.039680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b50e81-f58f-4e5a-ae81-bcbcf9c9bd70 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:34.072296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48034b-9b8a-4f06-bb65-7fc68dd10ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:34.073175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48034b-9b8a-4f06-bb65-7fc68dd10ddb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:34.073462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48034b-9b8a-4f06-bb65-7fc68dd10ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:34.073747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48034b-9b8a-4f06-bb65-7fc68dd10ddb None admin] PATH_INFO: `/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:34.074925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48034b-9b8a-4f06-bb65-7fc68dd10ddb None admin] RBAC: Authorizing `identity:get_project(project_id=40507a9aaa6a4fb681b9a91b92f9d369)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:34.076414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48034b-9b8a-4f06-bb65-7fc68dd10ddb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:34.078830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 568/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:33 2022] GET /identity/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:34.085047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:34.182661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bfd7-6dcc-41fe-9d29-749d030835af None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:34.205415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:34.216712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87b328-b21c-43c9-8b98-805474d9ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:34.217665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87b328-b21c-43c9-8b98-805474d9ed7c None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:34.217946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87b328-b21c-43c9-8b98-805474d9ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:34.218220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87b328-b21c-43c9-8b98-805474d9ed7c None admin] PATH_INFO: `/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369/users/5a13bd21079d486db22383667db1d531/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:34.220151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87b328-b21c-43c9-8b98-805474d9ed7c None admin] RBAC: Authorizing `identity:create_grant(project_id=40507a9aaa6a4fb681b9a91b92f9d369, user_id=5a13bd21079d486db22383667db1d531, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:34.222646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87b328-b21c-43c9-8b98-805474d9ed7c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:34.237164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 585/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:32:34 2022] PUT /identity/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369/users/5a13bd21079d486db22383667db1d531/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:32:36.411460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff5185-39f0-4f1a-8289-7fd620c07c79 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:36.412015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff5185-39f0-4f1a-8289-7fd620c07c79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:36.412015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff5185-39f0-4f1a-8289-7fd620c07c79 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:36.413435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 569/1154] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:36.418710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:36.419135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:36.419276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:36.436726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:36.437936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:36.509960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 586/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:36.533985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff5185-39f0-4f1a-8289-7fd620c07c79 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:36.591545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:36.635372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff5185-39f0-4f1a-8289-7fd620c07c79 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:36.673929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:36.674802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:36.675094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:36.675381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:36.686816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:36.694456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:36.695580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:36.696682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 570/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:36.702524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:36.799579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb7ec6-d0d8-4acc-bc6d-c7803dfa98f5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:36.813255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:36.835538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:36.836213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:36.836533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:36.836810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:36.837560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:36.837869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:36.839246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:36.848209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 587/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:36.854014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:36.955105 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3599b0-fa47-4821-898f-5204d5c300c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:36.970129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:36.990470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:36.991342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:36.991623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:36.991898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] PATH_INFO: `/v3/users/5a13bd21079d486db22383667db1d531` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:36.993119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] RBAC: Authorizing `identity:get_user(user_id=5a13bd21079d486db22383667db1d531)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:36.994562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:36.996187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 571/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:36 2022] GET /identity/v3/users/5a13bd21079d486db22383667db1d531 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:37.002205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:37.102900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86016356-8f4c-4cef-96da-3afb10a81b1c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:37.136282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:37.136946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:37.137228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:37.137502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:37.182264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:37.205945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:37.207476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:37.304267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 588/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:37.310384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:37.417221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b1935-e492-4777-983e-0f7917ddc710 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:37.452556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda5da9-c5fe-467c-aed7-db960ab64cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:37.453427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda5da9-c5fe-467c-aed7-db960ab64cca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:37.453709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda5da9-c5fe-467c-aed7-db960ab64cca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:37.453990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda5da9-c5fe-467c-aed7-db960ab64cca None admin] PATH_INFO: `/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:37.455181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda5da9-c5fe-467c-aed7-db960ab64cca None admin] RBAC: Authorizing `identity:get_project(project_id=40507a9aaa6a4fb681b9a91b92f9d369)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:37.456712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda5da9-c5fe-467c-aed7-db960ab64cca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:37.458641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 572/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:37 2022] GET /identity/v3/projects/40507a9aaa6a4fb681b9a91b92f9d369 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:37.464947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:37.546592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:37.570172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92343e7b-6f8b-443f-978e-122d194ff693 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:37.603068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:37.603721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:37.604007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:37.604286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:37.605134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=40507a9aaa6a4fb681b9a91b92f9d369, user.id=5a13bd21079d486db22383667db1d531) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:37.605450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:37.606882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1cf97-d83a-4cb1-86d3-459861bbde7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:37.613272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 589/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:37 2022] GET /identity/v3/role_assignments?user.id=5a13bd21079d486db22383667db1d531&scope.project.id=40507a9aaa6a4fb681b9a91b92f9d369&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:39.841731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dcee8-9ca9-4322-83c3-16f2ff5abe9c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:39.842318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dcee8-9ca9-4322-83c3-16f2ff5abe9c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:39.842318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dcee8-9ca9-4322-83c3-16f2ff5abe9c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:39.843678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 573/1162] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:39.849928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:39.850391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:39.850596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:39.867461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:39.868706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:39.936527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 590/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:39.959546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dcee8-9ca9-4322-83c3-16f2ff5abe9c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:40.013511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:40.066950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dcee8-9ca9-4322-83c3-16f2ff5abe9c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:40.101413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:40.101979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:40.102267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:40.102552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:40.169953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:40.171665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:40.221737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:40.276710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 574/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:40.282977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:40.363900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:40.398395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6077c76-8532-4624-adc9-53f9ec6f2ce4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:40.437493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:40.438313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:40.438602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:40.438882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:40.446771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:40.448243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0b5c214d-32c7-4010-a5b6-94c2af974fc5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:40.463900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:40.465246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:40.466606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 591/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:32:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:40.471463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:40.584065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77cdc-441d-46a1-8f59-b4db18d67e1f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:40.622731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:40.623366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:40.623651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:40.623934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:40.624712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:40.625029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:40.626453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4fe8e-df6b-4c6a-aa57-009932379e09 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:40.635735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 575/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu May 26 13:32:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:42.857719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c9829-6f6d-4112-b4a0-4a842a0815cc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:42.858262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c9829-6f6d-4112-b4a0-4a842a0815cc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:42.858454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c9829-6f6d-4112-b4a0-4a842a0815cc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:42.859847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 592/1167] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:42.865649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d3dc-d987-4444-b5ee-6940a89120d5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:42.866104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d3dc-d987-4444-b5ee-6940a89120d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:42.866216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d3dc-d987-4444-b5ee-6940a89120d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:42.883665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e408d3dc-d987-4444-b5ee-6940a89120d5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:42.885606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e408d3dc-d987-4444-b5ee-6940a89120d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:42.895346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:42.961151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 576/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:42.984922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c9829-6f6d-4112-b4a0-4a842a0815cc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:43.085605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c9829-6f6d-4112-b4a0-4a842a0815cc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:43.113786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:43.119816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b176cc2-b467-4c4f-9d4e-02270e090ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:43.120556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b176cc2-b467-4c4f-9d4e-02270e090ab5 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:43.120880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b176cc2-b467-4c4f-9d4e-02270e090ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:43.121196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b176cc2-b467-4c4f-9d4e-02270e090ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:43.121974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b176cc2-b467-4c4f-9d4e-02270e090ab5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:43.123902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b176cc2-b467-4c4f-9d4e-02270e090ab5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:43.137640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 593/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu May 26 13:32:42 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:32:45.442347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c681a-2a66-4c3f-83dc-bcd8f4dc8d1f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:45.442954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c681a-2a66-4c3f-83dc-bcd8f4dc8d1f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:45.442954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c681a-2a66-4c3f-83dc-bcd8f4dc8d1f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:45.444274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 577/1170] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:45.450569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:45.451051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:45.451195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:45.472702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:45.474358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:45.549480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 594/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:45.573734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c681a-2a66-4c3f-83dc-bcd8f4dc8d1f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:45.691136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c681a-2a66-4c3f-83dc-bcd8f4dc8d1f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:45.733470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5808e7d-7e07-4731-8384-ff4714cc22a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:45.734253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5808e7d-7e07-4731-8384-ff4714cc22a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:45.734532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5808e7d-7e07-4731-8384-ff4714cc22a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:45.734811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5808e7d-7e07-4731-8384-ff4714cc22a8 None admin] PATH_INFO: `/v3/domains/f14d47b7057f4506925e49ccca4a281b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:45.746018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5808e7d-7e07-4731-8384-ff4714cc22a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f14d47b7057f4506925e49ccca4a281b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:45.748150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5808e7d-7e07-4731-8384-ff4714cc22a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:45.749939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 578/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu May 26 13:32:45 2022] GET /identity/v3/domains/f14d47b7057f4506925e49ccca4a281b => generated 267 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:32:45.755915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:45.858265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6062a72-548f-4b17-ad2e-2cbb643dc0ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:45.878584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:45.894399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2608037-0f12-4fcb-8f3e-9e18576a8283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:45.895071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608037-0f12-4fcb-8f3e-9e18576a8283 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:45.895355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608037-0f12-4fcb-8f3e-9e18576a8283 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:45.895630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608037-0f12-4fcb-8f3e-9e18576a8283 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:45.896364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2608037-0f12-4fcb-8f3e-9e18576a8283 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:45.897799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2608037-0f12-4fcb-8f3e-9e18576a8283 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:45.926232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 595/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:32:45 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:32:48.180237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e34541-55b1-444b-b542-8149848599b1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:48.180770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e34541-55b1-444b-b542-8149848599b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:48.180770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e34541-55b1-444b-b542-8149848599b1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:48.182248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 579/1174] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:48.188429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:48.188874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:48.188983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:48.205809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:48.207000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:48.274453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 596/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:48.299859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e34541-55b1-444b-b542-8149848599b1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:48.303673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:48.401255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e34541-55b1-444b-b542-8149848599b1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:48.435151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52e68f-e176-4f8d-a671-616926f83517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:48.436006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e68f-e176-4f8d-a671-616926f83517 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:48.436290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e68f-e176-4f8d-a671-616926f83517 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:48.436595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e68f-e176-4f8d-a671-616926f83517 None admin] PATH_INFO: `/v3/domains/f14d47b7057f4506925e49ccca4a281b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:48.437771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e68f-e176-4f8d-a671-616926f83517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f14d47b7057f4506925e49ccca4a281b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:48.439221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e68f-e176-4f8d-a671-616926f83517 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:48.440874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 580/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu May 26 13:32:48 2022] GET /identity/v3/domains/f14d47b7057f4506925e49ccca4a281b => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:48.446646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:48.560578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac5c9a-7c49-4eae-ba8d-5f308b031812 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:48.562218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:48.603403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f7699-ccb5-491d-9976-cba4634d19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:48.604074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f7699-ccb5-491d-9976-cba4634d19ce None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:48.604378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f7699-ccb5-491d-9976-cba4634d19ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:48.604656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f7699-ccb5-491d-9976-cba4634d19ce None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:48.605372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f7699-ccb5-491d-9976-cba4634d19ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:48.606798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f7699-ccb5-491d-9976-cba4634d19ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:48.629493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 597/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:32:48 2022] POST /identity/v3/users => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:32:50.888068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b163-c366-4985-9175-80b14026edba None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:50.888720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b163-c366-4985-9175-80b14026edba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:50.888921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b163-c366-4985-9175-80b14026edba None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:50.890739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 581/1178] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:50.896251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d85f1-741d-457b-b248-7c4651337322 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:50.896788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d85f1-741d-457b-b248-7c4651337322 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:50.896883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d85f1-741d-457b-b248-7c4651337322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:50.914037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-125d85f1-741d-457b-b248-7c4651337322 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:50.915232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-125d85f1-741d-457b-b248-7c4651337322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:50.985770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 598/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:32:51.012798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b163-c366-4985-9175-80b14026edba None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:51.090707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:51.126260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b163-c366-4985-9175-80b14026edba None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:51.162260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:51.162977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:51.163262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:51.163540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:51.174490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:51.182136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:51.183495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:51.185076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 582/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:51.190106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d85f1-741d-457b-b248-7c4651337322 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:51.287637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:51.308960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d85f1-741d-457b-b248-7c4651337322 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:51.345376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:51.346056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:51.346341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:51.346622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:51.347379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:51.347690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:51.349126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:51.365386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 599/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:51.371280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:51.473307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84520188-88c9-48b0-847d-78ce3c92cc93 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:51.502792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:51.509136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:51.510013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:51.510288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:51.510557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] PATH_INFO: `/v3/users/68117a691f784474b51fc00899184382` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:51.524342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] RBAC: Authorizing `identity:get_user(user_id=68117a691f784474b51fc00899184382)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:51.525802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:51.527493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 583/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:51 2022] GET /identity/v3/users/68117a691f784474b51fc00899184382 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:51.534273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:51.641024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75857e-9d69-4efb-a472-33529cfd15b3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:51.679166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:51.679819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:51.680107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:51.680419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:51.699948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:51.756587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:51.757982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:51.854830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 600/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:51.860951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:51.958907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f267-1728-46a2-a5db-8085d23d083f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:51.992956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf7980-92fa-496a-bd6c-098ab6ff038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:51.993829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf7980-92fa-496a-bd6c-098ab6ff038f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:51.994112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf7980-92fa-496a-bd6c-098ab6ff038f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:51.994392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf7980-92fa-496a-bd6c-098ab6ff038f None admin] PATH_INFO: `/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:51.995591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf7980-92fa-496a-bd6c-098ab6ff038f None admin] RBAC: Authorizing `identity:get_project(project_id=cbdb46b1f92c47689b380e4ae7fc5f60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:51.997072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf7980-92fa-496a-bd6c-098ab6ff038f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:51.999110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 584/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:51 2022] GET /identity/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:52.005300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:52.054601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:52.104009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea3bbe-474d-4122-8fd4-fcffe91c24cd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:52.137353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:52.138013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:52.138882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:52.139161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:52.139993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=cbdb46b1f92c47689b380e4ae7fc5f60, user.id=68117a691f784474b51fc00899184382) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:52.140332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:52.141780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2527c6c-a91c-49db-8288-843d199d943c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:52.147525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 601/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:52 2022] GET /identity/v3/role_assignments?user.id=68117a691f784474b51fc00899184382&scope.project.id=cbdb46b1f92c47689b380e4ae7fc5f60&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:54.451465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe698f7-1a00-40c4-90fc-8d0f3a01fc37 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:54.452005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe698f7-1a00-40c4-90fc-8d0f3a01fc37 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:54.452005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe698f7-1a00-40c4-90fc-8d0f3a01fc37 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:54.453452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 585/1186] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:54.460126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:54.460646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:54.460738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:54.478538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:54.479755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:54.548820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 602/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:54.571908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe698f7-1a00-40c4-90fc-8d0f3a01fc37 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:54.634311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:54.695937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe698f7-1a00-40c4-90fc-8d0f3a01fc37 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:54.737572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:54.738269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:54.738554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:54.738833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:54.750252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:54.758111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:54.759715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:32:54.761447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 586/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:54.765826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:54.815659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:54.876544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a0e13-e0fa-4036-bfe0-f29addb88a36 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:54.913705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:54.914776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:54.915393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:54.915709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:54.916584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:54.916903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:54.921848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:54.931103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 603/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:54.938098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:55.047873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025c3ad-8844-4c27-bdfa-05186aab20c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:55.069156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:55.093496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:55.094385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:55.094677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:55.094950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] PATH_INFO: `/v3/users/68117a691f784474b51fc00899184382` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:55.096197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] RBAC: Authorizing `identity:get_user(user_id=68117a691f784474b51fc00899184382)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:55.097693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:55.099394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 587/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:54 2022] GET /identity/v3/users/68117a691f784474b51fc00899184382 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:55.105572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:55.210398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd81e3-f49b-491c-842c-a5113a062413 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:55.218796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:55.246702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761edb-d30c-452a-9b19-4eceac04342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:55.247694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761edb-d30c-452a-9b19-4eceac04342a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:55.248107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761edb-d30c-452a-9b19-4eceac04342a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:55.248421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761edb-d30c-452a-9b19-4eceac04342a None admin] PATH_INFO: `/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:55.249689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed761edb-d30c-452a-9b19-4eceac04342a None admin] RBAC: Authorizing `identity:get_project(project_id=cbdb46b1f92c47689b380e4ae7fc5f60)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:55.251160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed761edb-d30c-452a-9b19-4eceac04342a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:55.253248 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 604/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:55 2022] GET /identity/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:55.260049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:55.370816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c9b43-0d1c-4641-b907-cf46552862dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:55.414999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbadc82-5c76-4aa0-a095-ced91b01a5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:55.415960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbadc82-5c76-4aa0-a095-ced91b01a5c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:55.416269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbadc82-5c76-4aa0-a095-ced91b01a5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:55.416790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbadc82-5c76-4aa0-a095-ced91b01a5c6 None admin] PATH_INFO: `/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60/users/68117a691f784474b51fc00899184382/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:55.419006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbadc82-5c76-4aa0-a095-ced91b01a5c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbdb46b1f92c47689b380e4ae7fc5f60, user_id=68117a691f784474b51fc00899184382, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:55.421338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbadc82-5c76-4aa0-a095-ced91b01a5c6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:55.438649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 588/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:32:55 2022] PUT /identity/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60/users/68117a691f784474b51fc00899184382/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:32:57.661645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b9fd-f6fc-4e6f-b476-c387c8d83db8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:57.662224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b9fd-f6fc-4e6f-b476-c387c8d83db8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:57.662224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b9fd-f6fc-4e6f-b476-c387c8d83db8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:57.663559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 605/1193] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:32:57.669976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:57.670521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:57.670625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:57.687488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:32:57.688734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:32:57.734883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:57.765169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 589/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:32:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:32:57.791234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b9fd-f6fc-4e6f-b476-c387c8d83db8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:57.897250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b9fd-f6fc-4e6f-b476-c387c8d83db8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:57.931073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:57.931861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:57.932147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:57.932455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:57.942372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:32:57.949848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:32:57.951839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:32:57.953086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 606/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:32:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:32:57.957840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:58.054252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2372-7bb7-4671-9834-42469e7ef05b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:58.087813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:58.088506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:58.088794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:58.089073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:58.089803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:58.090112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:58.092118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:58.100130 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 590/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:32:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:58.105943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:58.140764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:58.216216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a95a88-096a-47a4-b0bf-3c0884da2aa8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:58.249097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:58.249964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:58.250250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:58.250530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] PATH_INFO: `/v3/users/68117a691f784474b51fc00899184382` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:58.251716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] RBAC: Authorizing `identity:get_user(user_id=68117a691f784474b51fc00899184382)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:58.253421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:58.255334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 607/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu May 26 13:32:58 2022] GET /identity/v3/users/68117a691f784474b51fc00899184382 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:58.261186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:58.300234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:58.360144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212446f-fa1e-4a93-952d-962f2c90ebfb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:58.394160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:58.394800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:58.395083 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:58.395358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:58.462690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:58.464882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:58.508725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:58.567110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 591/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:32:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:32:58.573001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:58.670809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e8334-04aa-4343-be79-8bf3eab361da None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:58.672356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:32:58.705168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d246d-1944-45dc-a441-4d7df99f5f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:58.706018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d246d-1944-45dc-a441-4d7df99f5f78 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:58.706298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d246d-1944-45dc-a441-4d7df99f5f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:58.706570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d246d-1944-45dc-a441-4d7df99f5f78 None admin] PATH_INFO: `/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:58.707993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d246d-1944-45dc-a441-4d7df99f5f78 None admin] RBAC: Authorizing `identity:get_project(project_id=cbdb46b1f92c47689b380e4ae7fc5f60)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:58.709448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d246d-1944-45dc-a441-4d7df99f5f78 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:58.711125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 608/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu May 26 13:32:58 2022] GET /identity/v3/projects/cbdb46b1f92c47689b380e4ae7fc5f60 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:32:58.717744 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:32:58.822049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314e033-257a-4290-ba1a-83459cfcf34c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:32:58.857221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:32:58.857882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:32:58.858163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:32:58.858441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:32:58.859274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=cbdb46b1f92c47689b380e4ae7fc5f60, user.id=68117a691f784474b51fc00899184382) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:32:58.859598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:32:58.861207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be597f55-71ff-4e35-9026-f811593bc91a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:32:58.867363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 592/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:32:58 2022] GET /identity/v3/role_assignments?user.id=68117a691f784474b51fc00899184382&scope.project.id=cbdb46b1f92c47689b380e4ae7fc5f60&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:33:08.700331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=15 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:33:08.864599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:00.510559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a40fa-08a9-4241-85fb-6fc6a7a545a9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:00.513494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a40fa-08a9-4241-85fb-6fc6a7a545a9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:00.513494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a40fa-08a9-4241-85fb-6fc6a7a545a9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:00.515091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 609/1201] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:00.522755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:00.523302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:00.523596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:00.546514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:00.548128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:00.648414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 593/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:00.684945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a40fa-08a9-4241-85fb-6fc6a7a545a9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:00.844749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a40fa-08a9-4241-85fb-6fc6a7a545a9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:00.897369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:00.898714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:00.898955 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:00.899263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:00.920186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:00.922855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:00.952724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:00.953792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:00.954834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:00.954834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:00.954834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:00.954834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:00.954834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 610/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:00.960820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:01.103178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5680ecb-11c2-4160-b988-72d55f993391 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:01.145779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:01.153721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:01.154691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:01.155163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:01.155466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:01.156197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:01.156539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:01.158186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96bb2a-9893-4261-8e48-95dbe615c0a2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:01.184869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 594/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:01.187134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:01.259978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:01.323887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916947d-d466-4822-861b-437f19d0f2b7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:01.363470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848ca60-792c-4425-896d-b92b1cdb3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:01.364079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ca60-792c-4425-896d-b92b1cdb3dcd None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:01.364399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ca60-792c-4425-896d-b92b1cdb3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:01.364685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ca60-792c-4425-896d-b92b1cdb3dcd None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:01.365494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848ca60-792c-4425-896d-b92b1cdb3dcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:01.367027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848ca60-792c-4425-896d-b92b1cdb3dcd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:01.400847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 611/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu May 26 13:34:01 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:34:03.898091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2997e-9e42-438b-b090-6ae08d32ad36 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:03.898641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2997e-9e42-438b-b090-6ae08d32ad36 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:03.898641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2997e-9e42-438b-b090-6ae08d32ad36 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:03.900907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 595/1206] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:03.907709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:03.908252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:03.909430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:03.930184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:03.931607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:04.014493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 612/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:04.042323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2997e-9e42-438b-b090-6ae08d32ad36 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:04.170671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2997e-9e42-438b-b090-6ae08d32ad36 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:04.210465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:04.211287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:04.211566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:04.211942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:04.224991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:34:04.234161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:04.235445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:04.236892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 596/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:34:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:04.243103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:04.294368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:04.368353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dc17f-0bb4-4b22-8cc6-96dd6374ec54 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:04.414290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:04.415007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:04.415293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:04.415575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:04.417280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:04.417610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:04.419084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:04.429694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 613/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:34:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:04.436518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:04.506399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:04.569664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcef8b3-ca15-467f-bdaa-1b93899f3450 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:04.608865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:04.609676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:04.609957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:04.610230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:04.622547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:04.624358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:04.646108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:04.647139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:04.649380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:04.649380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:04.649380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:04.649380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:04.649380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 597/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:04.653783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:04.757908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:04.762558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4b948-7003-4f04-b0bf-1ed3f61f4681 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:04.802153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:04.802911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:04.803197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:04.803536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:04.804188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:04.804513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:04.805882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:04.819547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 614/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:04.825641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:04.950530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9c5d1-560d-4ef6-86b8-7ad28e33c981 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:04.952323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:04.991032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:04.991921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:04.992255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:04.992601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:05.001110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:05.002878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:34:05.021764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:05.023134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:05.024418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:05.028442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 598/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:05.029609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:05.149549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d26f9b-aaae-4926-b9f8-424f47905f2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:05.191016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:05.191882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:05.192175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:05.192557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:05.193514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:05.193816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:05.195593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:05.217796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 615/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu May 26 13:34:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:05.224060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:05.352378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f096b-7e8a-425c-bf6d-dbd961bef91b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:05.392439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:05.393103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:05.393381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:05.393659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:05.427851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:05.478111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:05.480384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:05.606851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 599/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:34:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 13:34:05.608334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:05.628287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:05.718418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfdcdc-759d-4251-8dd9-e4fcf0d4b3f3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:05.756957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:05.758128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:05.758521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:05.758800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:05.770876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:05.772363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:05.792252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:05.793509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:05.799056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:05.799056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:05.799056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:05.799056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:05.799056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 616/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:34:05.799418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:05.873051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:05.918111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fe8ee-2272-4aea-9f47-14f129472f85 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:05.960921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:05.961620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:05.961881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:05.962147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:05.962913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:05.963220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:05.964707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:05.977208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 600/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:34:05.982728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:06.060177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:06.093869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00d616-4274-43dd-ac9e-77fb339fb373 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:06.134372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:06.136121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:06.136524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:06.136971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:06.144769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:06.146846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:34:06.163143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:06.164435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:06.165593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:06.167877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 617/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:06.173829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:06.294847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f3bd-3fc3-4f42-9877-257fec439b32 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:06.320091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:06.334764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:06.335513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:06.335868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:06.336353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:06.337181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:06.337518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:06.339360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336e528-e5ea-45ee-92ec-c2b09441a367 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:06.360345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 601/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:34:06.362426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:06.485346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c36f32-f7f3-43b9-88f3-ba1968ba3f3c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:06.519260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:06.527126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:06.527871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:06.528250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:06.528627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:06.529547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=f14da5405ccf4fdfaf91b1ec52ad3801) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:06.529925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:06.531495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be01beb-21ba-45e2-bd72-2c3156458513 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:06.552274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 618/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:34:06 2022] GET /identity/v3/role_assignments?user.id=f14da5405ccf4fdfaf91b1ec52ad3801&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:09.154647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0be3-540f-4d86-b4cf-aa2b9a9758e4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:09.155180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0be3-540f-4d86-b4cf-aa2b9a9758e4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:09.155180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0be3-540f-4d86-b4cf-aa2b9a9758e4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:09.156615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 602/1220] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:09.161918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:09.162376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:09.162551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:09.184188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:09.185416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:09.265945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 619/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:09.290363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0be3-540f-4d86-b4cf-aa2b9a9758e4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:09.408503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0be3-540f-4d86-b4cf-aa2b9a9758e4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:09.447644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:09.448608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:09.449063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:09.449354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:09.461081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:34:09.469299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:09.470505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:09.472635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 603/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:34:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:09.477194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:09.589825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cbff-cf9e-438b-be8c-7fad04505cb7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:09.617566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:09.630899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:09.631648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:09.631989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:09.632322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:09.633178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:09.633554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:09.634962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:09.643326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 620/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:34:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:09.649337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:09.699020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:09.768281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764020ee-6a3c-41a3-891c-91170c23d8c1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:09.810304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:09.811106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:09.811395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:09.811679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:09.822635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:09.824189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:09.843936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:09.845173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:09.846148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:09.846148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:09.846148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:09.846148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:09.846148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 604/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:09.851161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:09.974006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fa471-241a-42b1-93f4-0ba8e9d6bc85 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:10.008398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:10.009075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:10.009923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:10.010192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:10.011130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:10.011483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:10.012872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:10.025704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 621/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:10.032008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:10.146797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:10.155772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa0fd0-f7e8-4fc0-b66f-bd6518d2b574 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:10.195684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:10.197419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:10.197901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:10.198336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:10.208531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:10.210037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:34:10.227517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:10.228737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:10.229930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 605/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:10.237704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:10.294366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:10.362098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29484063-a2b9-4267-b118-03249dc1c008 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:10.405515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:10.406321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:10.406749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:10.407031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:10.407919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:10.408231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:10.409784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:10.430389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 622/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu May 26 13:34:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:34:10.436736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:10.571928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b6c51-a723-4322-920d-25e772bda347 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:10.619829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:10.620951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:10.621500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:10.621923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:10.644941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:10.646673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:10.674435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:10.675460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:10.676461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:10.676461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:10.676461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:10.676461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:10.676461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 606/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:10.684568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:10.818448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999e08-49b5-4f5c-ba91-6b33da84b3fc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:10.821580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:10.873464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:10.874360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:10.874737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:10.874918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:10.875773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:10.876209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:10.878458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:10.894867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 623/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:10.901760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:10.912349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:11.043318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab99d0-096f-4c7f-a173-5d9413a68f48 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:11.097313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:11.098315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:11.098630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:11.099022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:11.110527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:11.112227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:34:11.129916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:11.136705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:11.137871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 607/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:34:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:34:11.138503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:11.267922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed045e-e2b8-44d0-9616-0aa3d11749a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:11.304259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:11.305283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:11.305606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:11.305892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:11.307365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:11.307675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:11.309159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129310dd-597e-45d3-a302-d1ec5e07b5a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:11.323075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 624/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:34:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:34:11.329152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:11.400289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:11.443395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132de276-3dfc-4e37-a574-80a5faf54ece None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:11.483293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d187def-7746-4465-84a3-90e6770d2e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:11.484191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187def-7746-4465-84a3-90e6770d2e71 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:11.484507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187def-7746-4465-84a3-90e6770d2e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:11.484783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187def-7746-4465-84a3-90e6770d2e71 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/f14da5405ccf4fdfaf91b1ec52ad3801/roles/c69be4541efc4e588a03e942f9aa5bbb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:11.499832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d187def-7746-4465-84a3-90e6770d2e71 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=f14da5405ccf4fdfaf91b1ec52ad3801, role_id=c69be4541efc4e588a03e942f9aa5bbb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:11.501317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d187def-7746-4465-84a3-90e6770d2e71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:11.516246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 608/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:34:11 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/f14da5405ccf4fdfaf91b1ec52ad3801/roles/c69be4541efc4e588a03e942f9aa5bbb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:34:13.906314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c6959-54fc-40a0-b41d-814dab7882ad None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:13.906873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c6959-54fc-40a0-b41d-814dab7882ad None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:13.906873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c6959-54fc-40a0-b41d-814dab7882ad None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:13.908255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 625/1233] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:13.914098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:13.914595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:13.914693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:13.935305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:13.936684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:14.013197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 609/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:14.039305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c6959-54fc-40a0-b41d-814dab7882ad None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:14.067897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:14.165056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c6959-54fc-40a0-b41d-814dab7882ad None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:14.207473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:14.209193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:14.209671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:14.210092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:14.227218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:34:14.243991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:14.245189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:14.246240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 626/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu May 26 13:34:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:14.255727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:14.311220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:14.375365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c7baf-69bd-44b2-9597-356f5613d16a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:14.412374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:14.413305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:14.413588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:14.413867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:14.414606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:14.414912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:14.417013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:14.430786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 610/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu May 26 13:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:14.432204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:14.535459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:14.554865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f579024-f886-46b1-8933-51fb55dc7629 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:14.601921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:14.602738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:14.603596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:14.604121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:14.616726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:14.618367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:14.638998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:14.640154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:14.641263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:14.641263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:14.641263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:14.641263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:14.641263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 627/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:14.646732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:14.767261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c9dfa-e70e-4194-ae1e-ef509f7d0e67 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:14.777938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:14.807080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:14.807968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:14.808273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:14.808581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:14.809430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:14.809777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:14.811337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:14.826038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 611/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:14.832427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:14.947810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b800f43-99dd-4ad6-b5fe-c50149bc3dd1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:14.985657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:14.986503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:14.986787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:14.987063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:14.990269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:15.000224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:15.002053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:34:15.022502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:15.023780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:15.024872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 628/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:34:15.030908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:15.146993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67c9e0-240d-4b54-a3a5-9472a13cf063 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:15.188534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:15.189308 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:15.189595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:15.189872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:15.190699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:15.191036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:15.196636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:15.211415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 612/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu May 26 13:34:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:15.219501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:15.330600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d89c-81e0-48e6-93b0-51f4760d0b9f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:15.379552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:15.380209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:15.380530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:15.380810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:15.444830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:15.453340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:15.454813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:15.564717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 629/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:34:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:34:15.571623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:15.603776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:15.689702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c921886-d801-4c0d-b981-bf447c6f0f75 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:15.731786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:15.732742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:15.733128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:15.733419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:15.747358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:15.749120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:15.770966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:15.772129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:15.773148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:15.773148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:15.773148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:15.773148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:15.773148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 613/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:15.778776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:15.891293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:15.898330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e983d-059d-484d-8586-7a17ea6d4900 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:15.941244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:15.941957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:15.942325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:15.942606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:15.943333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:15.943635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:15.945138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:15.959723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 630/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:15.966298 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:16.053941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:16.079456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f66ad-d697-4acf-ba41-e52bccd6bdf3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:16.121125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:16.122038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:16.122327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:16.122610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:16.131306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:16.132846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:34:16.150345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:16.151512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:16.152758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 614/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:16.159096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:16.277347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a72c-1d29-4b7a-ac20-0478db43246a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:16.320528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:16.321208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:16.322138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:16.322415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:16.323217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:16.323774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:16.325664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a7d82-8dd2-4be0-aa92-28bf58bd48d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:16.338643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 631/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:16.345339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:16.470955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadd8c4-e8c2-437c-b120-1243aa18f4d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:16.507687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:16.508396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:16.508685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:16.509016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:16.510996 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] RBAC: Adding query filter params (role.id=c69be4541efc4e588a03e942f9aa5bbb, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=f14da5405ccf4fdfaf91b1ec52ad3801) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:16.511265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:16.512911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc033cd-5e56-4097-885c-3a727c8869b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:16.528979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 615/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:34:16 2022] GET /identity/v3/role_assignments?user.id=f14da5405ccf4fdfaf91b1ec52ad3801&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=c69be4541efc4e588a03e942f9aa5bbb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:18.966258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a17589-45e2-4439-a0dc-0565ae82fa23 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:18.966924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a17589-45e2-4439-a0dc-0565ae82fa23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:18.966924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a17589-45e2-4439-a0dc-0565ae82fa23 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:18.968292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 632/1247] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:18.973909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f67c-991d-4535-a623-73765e242a94 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:18.974500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f67c-991d-4535-a623-73765e242a94 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:18.974500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f67c-991d-4535-a623-73765e242a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:18.980351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:18.994872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2924f67c-991d-4535-a623-73765e242a94 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:18.996780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2924f67c-991d-4535-a623-73765e242a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:19.080141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 616/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 13:34:19.109449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a17589-45e2-4439-a0dc-0565ae82fa23 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:19.131359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:19.245988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a17589-45e2-4439-a0dc-0565ae82fa23 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:19.285910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:19.286786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:19.287236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:19.287658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:19.299758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:34:19.307565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:19.308800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:19.312574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 633/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:34:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:19.315958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f67c-991d-4535-a623-73765e242a94 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:19.433252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f67c-991d-4535-a623-73765e242a94 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:19.471095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:19.477559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:19.478874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:19.479156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:19.479435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:19.480180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:19.480523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:19.482475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:19.490926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 617/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:34:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:19.497211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:19.595237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:19.624006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692e58b-9ccc-47d5-aaa0-dcec18e0974f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:19.662750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:19.663549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:19.663833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:19.664106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:19.675517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:19.677012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:19.694755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:19.696184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:19.697892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:19.697892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:19.697892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:19.697892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:19.697892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 634/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:19.702688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:19.816704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a957c87-4e4c-4362-a820-cc3e619592b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:19.857915 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:19.858623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:19.858906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:19.859242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:19.860058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:19.860403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:19.862046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:19.877947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 618/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:19.884784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:20.010712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260c6df-7701-4b06-b8b3-f17e0301a842 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:20.043417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:20.053489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:20.054290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:20.054574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:20.054855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:20.064331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:20.065805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:34:20.083589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:20.084765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:20.085862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:20.090942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 635/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:20.091888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:20.157682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:20.204789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050cfc-ea91-4c06-b450-f8ef639d3fc1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:20.241288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:20.242133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:20.242496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:20.242817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:20.243815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:20.244127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:20.245933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:20.267095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 619/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu May 26 13:34:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:20.274129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:20.390683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9f762-004d-4fe0-a51f-c901e9163af7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:20.431794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:20.432689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:20.433095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:20.433418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:20.498872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:20.513497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:20.514873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:20.641804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 636/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:34:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:34:20.649172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:20.780081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1977da-3a51-4fe5-854c-04d5057fe969 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:20.794064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:20.820506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:20.822147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:20.823426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:20.823757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:20.837002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:20.838497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:20.858935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:20.860127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:20.861183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:20.861183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:20.861183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:20.861183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:20.861183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 620/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:20.866114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:20.965293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:20.979846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec05fc-bfdc-4322-baa4-8d3b7527e00c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:21.017827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:21.018496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:21.018782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:21.019063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:21.019817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:21.020116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:21.021580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:21.035352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 637/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:21.042290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:21.176938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa283b-02a8-4124-ba8f-0026f0fa1079 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:21.215561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:21.216413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:21.216704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:21.216986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:21.224465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:21.225959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:34:21.241409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:21.242563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:21.244540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 621/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:34:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:21.251402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:21.363068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4f9de-ea61-40f2-9967-073dc2626d39 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:21.398528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:21.399196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:21.399522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:21.399817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:21.400620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:21.400932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:21.402368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda7db8-bb7c-4eb8-a599-933af2005b7c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:21.414705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 638/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:34:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:21.421639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:21.450595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:21.563604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e2caa-3455-4501-95d8-76b00a8f9750 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:21.611207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:21.611917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:21.612208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:21.612560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:21.613398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=f14da5405ccf4fdfaf91b1ec52ad3801) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:21.613756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:21.615270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0418d-d68e-4109-8872-bcd6ae907663 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:21.622254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 622/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:34:21 2022] GET /identity/v3/role_assignments?user.id=f14da5405ccf4fdfaf91b1ec52ad3801&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:23.841065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d2e0-c6aa-4a20-8fe7-2d3d79717bcd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:23.841602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d2e0-c6aa-4a20-8fe7-2d3d79717bcd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:23.841690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d2e0-c6aa-4a20-8fe7-2d3d79717bcd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:23.842997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 639/1261] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:23.848237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:23.848723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:23.848818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:23.866784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:23.868546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:23.934787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 623/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:23.958346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7d2e0-c6aa-4a20-8fe7-2d3d79717bcd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:23.969407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:24.059485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7d2e0-c6aa-4a20-8fe7-2d3d79717bcd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:24.093490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:24.094205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:24.094542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:24.094824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:24.104848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:34:24.112234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:24.113333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:24.114221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 640/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:34:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:24.120147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:24.138678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:24.218803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9b8d-359a-4dd6-8695-a7cf72383773 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:24.252576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:24.253246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:24.253535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:24.253819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:24.254562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:24.254877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:24.256257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:24.265448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 624/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:34:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:24.271456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:24.337672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:24.372516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f0fd-2d8f-4263-9387-84528be35f7d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:24.404802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:24.405583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:24.405864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:24.406136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:24.417080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:24.418451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:24.436102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:24.437264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:24.438559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:24.438559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:24.438559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:24.438559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:24.438559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 641/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:24.443094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:24.521489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:24.540390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d8be6-401f-4434-9db3-a65ca8a2d58e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:24.577591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:24.578462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:24.578599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:24.578918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:24.579663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:24.579976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:24.581674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:24.595100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 625/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:24.601531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:24.719017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ef7b3-47c0-4052-ac96-d1c56da82844 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:24.739451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:24.753857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:24.754661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:24.754946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:24.755225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:24.762976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:24.764455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:34:24.779045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:24.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:24.781251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 642/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:24.786543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:24.897036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfacf39-aaa0-4ad5-8ef3-cc4530cd90cf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:24.934970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:24.935637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:24.935921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:24.936200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:24.937065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:24.937376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:24.938822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:24.948072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:24.957972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 626/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu May 26 13:34:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:24.964499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:25.061004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613141c7-7fae-4454-8e61-5511f799a528 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:25.093831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:25.094603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:25.094884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:25.095155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:25.107069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:25.108541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:25.125084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:25.126198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:25.127332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:25.127332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:25.127332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:25.127332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:25.127332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 643/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 13:34:25.132031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:25.232715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932007b-12df-4561-a6c2-19bf6f696dd7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:25.265933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:25.266585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:25.266862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:25.267139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:25.267844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:25.268148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:25.269546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:25.281144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 627/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:25.286815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:25.318805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:25.393908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8152-5d34-462d-beca-8bf44ead210e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:25.425928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:25.426721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:25.427008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:25.427288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:25.434413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:25.435881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:34:25.450535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:25.451658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:25.452698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 644/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:34:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:25.457999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:25.515484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:25.557577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963aa03-1d13-4a35-b31e-f29b664413b6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:25.593479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:25.594066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:25.594352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:25.594633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:25.595410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:25.595720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:25.597184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685da5-a201-4ed2-bc9a-d6c0b864b3fb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:25.609215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 628/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:34:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:25.615320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:25.706393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:25.715448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb47e-b99f-4781-9bc4-65ee9082dcc4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:25.750251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8571ed-d0fa-4d7c-ae2f-313206029cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:25.751147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8571ed-d0fa-4d7c-ae2f-313206029cf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:25.751434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8571ed-d0fa-4d7c-ae2f-313206029cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:25.751712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8571ed-d0fa-4d7c-ae2f-313206029cf8 None admin] PATH_INFO: `/v3/projects/07dccb3049524e049519153329f36e8f/users/f14da5405ccf4fdfaf91b1ec52ad3801/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:25.753888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8571ed-d0fa-4d7c-ae2f-313206029cf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=07dccb3049524e049519153329f36e8f, user_id=f14da5405ccf4fdfaf91b1ec52ad3801, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:25.755337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8571ed-d0fa-4d7c-ae2f-313206029cf8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:25.768939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 645/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu May 26 13:34:25 2022] PUT /identity/v3/projects/07dccb3049524e049519153329f36e8f/users/f14da5405ccf4fdfaf91b1ec52ad3801/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:34:27.963859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15341057-00af-47f0-ba0a-518ff1e408ed None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:27.964460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15341057-00af-47f0-ba0a-518ff1e408ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:27.964460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15341057-00af-47f0-ba0a-518ff1e408ed None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:27.965832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 629/1274] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:27.971050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:27.971461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:27.971608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:27.988805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:27.989996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:28.054407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 646/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:28.078101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15341057-00af-47f0-ba0a-518ff1e408ed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:28.178149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15341057-00af-47f0-ba0a-518ff1e408ed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:28.195685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:28.213795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:28.214589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:28.214877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:28.215159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:28.225091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 13:34:28.233318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:28.234527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:28.235457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 630/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu May 26 13:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:28.240855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:28.328507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:28.348864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e8198-67e1-42e3-be54-43a39f14f2db None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:28.387902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:28.388671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:28.388991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:28.389328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:28.390095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:28.390410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:28.391824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:28.401363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 647/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu May 26 13:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:34:28.407670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:28.529548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb9dd-0cc9-4eb2-943a-c5f8fe4083aa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:28.565963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:28.566968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:28.567295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:28.567605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:28.583314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:28.585503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:28.605499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:28.606631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:28.607728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:28.607728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:28.607728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:28.607728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:28.607728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 631/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:28.614857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:28.718519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758afa6-d699-48d1-bf49-9b71c7001714 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:28.751199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:28.751860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:28.752148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:28.752458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:28.753174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:28.753485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:28.754851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:28.767416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 648/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:28.773439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:28.804575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:28.876023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa638-2f01-4d9e-9459-466bc1a1ff02 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:28.910158 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:28.910966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:28.911248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:28.911526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:28.920329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:28.921778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 13:34:28.936074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:28.937282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:28.942959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 632/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:28.943606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:28.948069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:29.044870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2890-b91e-4850-87f2-4bc8e4fb3b24 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:29.078441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:29.079039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:29.079325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:29.079604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:29.080493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:29.081064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:29.082510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:29.099328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 649/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu May 26 13:34:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:29.105241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:29.194890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:29.205244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dca9c-9c08-4661-8f6b-55a124f48e82 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:29.238923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:29.239558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:29.239838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:29.240113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:29.308225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:29.309715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:29.396749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:29.407206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 633/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu May 26 13:34:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 13:34:29.413034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:29.475145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:29.515341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04370b92-abe1-4300-b274-d5600d2de2ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:29.555927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:29.556862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:29.557178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:29.557571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:29.572227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:29.572227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 13:34:29.591646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:29.592881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:29.594004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 13:34:29.594004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 13:34:29.594004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 13:34:29.594004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:29.594004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 650/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:29.598723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:29.699068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca4db7-5d9e-49fd-b56b-4c7b6bd3537c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:29.734107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:29.734775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:29.735063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:29.735341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:29.736065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:29.736417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:29.737811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:29.750037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 634/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu May 26 13:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:29.755923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:29.859988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4afd03-c135-4bf7-a57b-671f88058dd2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:29.874933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:29.897973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:29.899725 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:29.900174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:29.900507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:29.909330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:29.910456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 13:34:29.927998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:29.929059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 26 13:34:29.930115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 651/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu May 26 13:34:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:29.935556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:29.996623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:30.053816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ceb7dc-e9a3-4720-92f6-bfd16ff7ab5b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:30.091429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:30.092084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:30.092394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:30.092674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:30.093459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:30.093762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:30.095204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86255-07cc-4c30-8a3e-18257a6e2fb7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:30.107957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 635/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu May 26 13:34:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:30.115111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:30.218744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df79c-5cce-48e7-98ac-fe4ea73c7987 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:30.251840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:30.252597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:30.252884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:30.253159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:30.253984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] RBAC: Adding query filter params (role.id=2e85089f6c1a4bd1a54741d06e106e27, scope.project.id=07dccb3049524e049519153329f36e8f, user.id=f14da5405ccf4fdfaf91b1ec52ad3801) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:30.254295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:30.255898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046eeaff-c89f-4ab1-84a0-c59cc5b2377a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:30.263126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 652/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu May 26 13:34:30 2022] GET /identity/v3/role_assignments?user.id=f14da5405ccf4fdfaf91b1ec52ad3801&scope.project.id=07dccb3049524e049519153329f36e8f&role.id=2e85089f6c1a4bd1a54741d06e106e27 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:32.501783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2783bd1-d422-46aa-b62f-265fc8a68b79 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:32.502364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2783bd1-d422-46aa-b62f-265fc8a68b79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:32.502364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2783bd1-d422-46aa-b62f-265fc8a68b79 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:32.503719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 636/1288] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:32.509138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:32.509569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:32.509675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:32.526122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:32.527296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:32.557826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:32.606120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 653/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:32.634666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2783bd1-d422-46aa-b62f-265fc8a68b79 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:32.742625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2783bd1-d422-46aa-b62f-265fc8a68b79 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:32.753418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:32.777438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:32.778224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:32.778507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:32.778786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:32.779450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:32.780869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:32.792708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:32.793938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:32.794943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 637/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:34:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:32.799979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:32.899297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e04a8b-efe5-480f-b031-72f3214fab55 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:32.959334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:32.963126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:32.963420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:32.963704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:32.964469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:32.964786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:32.966190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d651a-ed84-42fa-a1f3-1fb195414534 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:32.972094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 654/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu May 26 13:34:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:32.978705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:33.081378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3b5f4-8f2c-40a6-9bfe-44192444b214 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:33.116746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:33.117422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:33.117740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:33.118024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:33.118743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:33.119056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:33.120490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433a25-8b1b-4f72-b79c-547ffd98417a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:33.125553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 638/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu May 26 13:34:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:35.357790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed312e4-db5f-4af0-bc4a-fd3a25a5c59a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:35.358327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed312e4-db5f-4af0-bc4a-fd3a25a5c59a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:35.358327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed312e4-db5f-4af0-bc4a-fd3a25a5c59a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:35.359734 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 655/1293] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:35.365156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa7658-eb7f-494f-8d95-ce408c2b9304 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:35.365600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa7658-eb7f-494f-8d95-ce408c2b9304 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:35.365725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa7658-eb7f-494f-8d95-ce408c2b9304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:35.382212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-41aa7658-eb7f-494f-8d95-ce408c2b9304 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:35.383525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-41aa7658-eb7f-494f-8d95-ce408c2b9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:35.412273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:35.450413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 639/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:35.473894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed312e4-db5f-4af0-bc4a-fd3a25a5c59a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:35.504272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:35.582316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed312e4-db5f-4af0-bc4a-fd3a25a5c59a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:35.626179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e5c3a-5d85-4073-9976-6eebe085805d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:35.626822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e5c3a-5d85-4073-9976-6eebe085805d None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:35.627115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e5c3a-5d85-4073-9976-6eebe085805d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:35.627399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e5c3a-5d85-4073-9976-6eebe085805d None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:35.628151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e5c3a-5d85-4073-9976-6eebe085805d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:35.629656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e5c3a-5d85-4073-9976-6eebe085805d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:35.646673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 656/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu May 26 13:34:35 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:34:37.905235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d685a-2bcd-46ca-b40f-f7788f872c97 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:37.905807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d685a-2bcd-46ca-b40f-f7788f872c97 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:37.905807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d685a-2bcd-46ca-b40f-f7788f872c97 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:37.907164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 640/1296] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:37.912417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:37.912826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:37.913012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:37.929665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:37.930860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:38.002717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 657/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:38.026741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d685a-2bcd-46ca-b40f-f7788f872c97 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:38.137435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d685a-2bcd-46ca-b40f-f7788f872c97 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:38.172501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:38.173222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:38.173549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:38.173824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:38.174537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:38.175964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:38.188899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:38.190195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:38.191053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 641/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:34:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:38.197279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:38.216742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:38.307093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94257-7cac-4cf9-bb8b-2c5739d2f441 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:38.346477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:38.347393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:38.347786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:38.348202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:38.349066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:38.349498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:38.351141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:38.358229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 658/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu May 26 13:34:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:38.364454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:38.402381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:38.480970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e48f14-e00f-4db7-9988-8573bde70f3f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:38.516443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:38.517108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:38.517393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:38.517670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:38.518460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] RBAC: Adding query filter params (interface=public, service_id=fb3aeb5d2538499a8b327b503b4f8a34, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:38.518772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:38.520178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a02054-ee6b-43b4-bb64-23e95513d635 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:38.526467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 642/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu May 26 13:34:38 2022] GET /identity/v3/endpoints?service_id=fb3aeb5d2538499a8b327b503b4f8a34&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:38.532142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:38.630643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:38.655160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0e6d-63c3-47cd-a14f-5fcc4f840f3d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:38.692059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fabdf-14c2-4ff6-9541-ef1a00d172d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:38.692776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fabdf-14c2-4ff6-9541-ef1a00d172d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:38.693050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fabdf-14c2-4ff6-9541-ef1a00d172d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:38.693327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fabdf-14c2-4ff6-9541-ef1a00d172d9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:38.694015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fabdf-14c2-4ff6-9541-ef1a00d172d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:38.696057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fabdf-14c2-4ff6-9541-ef1a00d172d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:38.703543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 659/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu May 26 13:34:38 2022] GET /identity/v3/services => generated 2191 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:34:40.967814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e923-01ff-4649-b261-ab175c081ba4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:40.968382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e923-01ff-4649-b261-ab175c081ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:40.968477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e923-01ff-4649-b261-ab175c081ba4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:40.969786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 643/1302] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:34:40.974960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:40.975345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:40.975523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:40.992712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:34:40.993904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:34:41.067388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 660/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:34:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 13:34:41.091104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e923-01ff-4649-b261-ab175c081ba4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:41.190243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:41.198485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e923-01ff-4649-b261-ab175c081ba4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:41.233879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:41.234686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:41.234973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:41.235257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:41.235918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:41.237364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 13:34:41.250034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 13:34:41.251317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 26 13:34:41.252619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 644/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu May 26 13:34:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 13:34:41.257648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:41.338475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:41.363374 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825cde5-9c68-43c0-bca7-2a2ba3c883d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:41.400529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:41.401211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:41.401495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:41.401772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:41.402489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:34:41.402796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:41.404245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74eee2-6235-403a-baac-39c11efbac0f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:41.410079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 661/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu May 26 13:34:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:34:41.417663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:34:41.531475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a402a2-4bfd-4116-80d8-9196e4b34f3c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:34:41.566867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c30b81-d36b-49aa-8b4e-f1651f7c728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:34:41.567534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30b81-d36b-49aa-8b4e-f1651f7c728f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:34:41.567815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30b81-d36b-49aa-8b4e-f1651f7c728f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:34:41.568092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30b81-d36b-49aa-8b4e-f1651f7c728f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:34:41.568862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c30b81-d36b-49aa-8b4e-f1651f7c728f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:34:41.570823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c30b81-d36b-49aa-8b4e-f1651f7c728f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:34:41.590569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 645/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu May 26 13:34:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:34:51.407156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:34:51.585267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:47.074198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3e4ce-a09a-4aac-8982-5585d7e67b99 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:47.074761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3e4ce-a09a-4aac-8982-5585d7e67b99 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:47.074761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3e4ce-a09a-4aac-8982-5585d7e67b99 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:47.076179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 662/1307] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:35:47.082073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58502443-20b2-4bfc-91e5-a4173c07f835 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:47.082600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58502443-20b2-4bfc-91e5-a4173c07f835 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:47.082600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58502443-20b2-4bfc-91e5-a4173c07f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:47.161776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-58502443-20b2-4bfc-91e5-a4173c07f835 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:35:47.163383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58502443-20b2-4bfc-91e5-a4173c07f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:47.270916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 646/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:35:47.320705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018dc1f-5a16-41fc-9047-3e3dee940f2b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:47.321092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018dc1f-5a16-41fc-9047-3e3dee940f2b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:47.321400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018dc1f-5a16-41fc-9047-3e3dee940f2b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:47.323718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 663/1309] 10.222.0.16 () {54 vars in 964 bytes} [Thu May 26 13:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:35:47.330839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a08f-50fe-45c9-a067-e9284d4c8ad1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:47.331136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a08f-50fe-45c9-a067-e9284d4c8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:47.331588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a08f-50fe-45c9-a067-e9284d4c8ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:47.420259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-baf3a08f-50fe-45c9-a067-e9284d4c8ad1 None None] MFA Rules not processed for user `6f44d10be1654d17990966ae64d43ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:35:47.421879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-baf3a08f-50fe-45c9-a067-e9284d4c8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:47.448107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:47.540817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 647/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Thu May 26 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:35:47.550101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018dc1f-5a16-41fc-9047-3e3dee940f2b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:35:47.552626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0018dc1f-5a16-41fc-9047-3e3dee940f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:47.692752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018dc1f-5a16-41fc-9047-3e3dee940f2b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:35:47.782503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:35:47.783209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:47.783555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:47.783936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:47.786431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:47.846894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:47.918577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:35:47.946414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14480e84-9923-4660-8136-ffb4d2bb8cee service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:35:47.950940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 664/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:35:50.555975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcdc51-7b7a-4fcf-a262-65171d9af064 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:50.556599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcdc51-7b7a-4fcf-a262-65171d9af064 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:50.556866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcdc51-7b7a-4fcf-a262-65171d9af064 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:50.564229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 648/1312] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:35:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:35:50.565821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93b9d-c9d6-4388-aa9f-46bb76f937a3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:50.566507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93b9d-c9d6-4388-aa9f-46bb76f937a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:50.567144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93b9d-c9d6-4388-aa9f-46bb76f937a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:50.677897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-42d93b9d-c9d6-4388-aa9f-46bb76f937a3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:35:50.679444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42d93b9d-c9d6-4388-aa9f-46bb76f937a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:50.803093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 665/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:35:50.857993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4321db-c7e2-433b-a37b-6c8c94318a87 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:50.858302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4321db-c7e2-433b-a37b-6c8c94318a87 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:50.858596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4321db-c7e2-433b-a37b-6c8c94318a87 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:50.860200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 649/1314] 10.222.0.16 () {54 vars in 964 bytes} [Thu May 26 13:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:35:50.868339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98998816-0ef2-4ae0-a6b8-d24f1bb072fd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:50.868867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98998816-0ef2-4ae0-a6b8-d24f1bb072fd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:50.869378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98998816-0ef2-4ae0-a6b8-d24f1bb072fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:50.888339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:50.996389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-98998816-0ef2-4ae0-a6b8-d24f1bb072fd None None] MFA Rules not processed for user `6f44d10be1654d17990966ae64d43ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:35:50.997638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-98998816-0ef2-4ae0-a6b8-d24f1bb072fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:51.084380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 666/1315] 10.222.0.16 () {58 vars in 1061 bytes} [Thu May 26 13:35:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:35:51.093318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4321db-c7e2-433b-a37b-6c8c94318a87 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:35:51.094845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9e4321db-c7e2-433b-a37b-6c8c94318a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:51.212262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4321db-c7e2-433b-a37b-6c8c94318a87 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:35:51.306009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:51.309445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:35:51.310092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:51.310986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:51.311344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:51.313758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:51.437600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:35:51.439180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd584d-5da5-4a76-9ce1-31f3069d6d34 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:35:51.468504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 650/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:35:54.266618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d91ea4-8516-4dba-a135-985d092d7ad3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:54.267156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d91ea4-8516-4dba-a135-985d092d7ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:54.267276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d91ea4-8516-4dba-a135-985d092d7ad3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:54.268941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 667/1317] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:35:54.275143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b962e3-6758-44d8-97c3-a1e010f6a866 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:54.275609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b962e3-6758-44d8-97c3-a1e010f6a866 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:54.275724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b962e3-6758-44d8-97c3-a1e010f6a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:54.341377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-18b962e3-6758-44d8-97c3-a1e010f6a866 None None] MFA Rules not processed for user `a2bb088e25f642dba7a1e5a300b90359`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:35:54.342602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18b962e3-6758-44d8-97c3-a1e010f6a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:54.394598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:54.441005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 651/1318] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:35:54.481628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d91ea4-8516-4dba-a135-985d092d7ad3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:35:54.483200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b0d91ea4-8516-4dba-a135-985d092d7ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:54.588747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d91ea4-8516-4dba-a135-985d092d7ad3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:35:54.591485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:54.661643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:35:54.662289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:54.662576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:54.662855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:54.664541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:54.770829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:35:54.772276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac674-ead6-4cde-a7b7-3acc3268ea60 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:35:54.790949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 668/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:35:57.593085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fb2b3-f6f8-4b3b-a2e4-8a13f655e9c7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:57.593651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fb2b3-f6f8-4b3b-a2e4-8a13f655e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:57.593651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fb2b3-f6f8-4b3b-a2e4-8a13f655e9c7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:57.595021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 652/1320] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:35:57.600529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ccde-caac-4e0f-be74-e34a0a71452b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:57.600993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ccde-caac-4e0f-be74-e34a0a71452b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:57.601141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ccde-caac-4e0f-be74-e34a0a71452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:57.639977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:57.667724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f355ccde-caac-4e0f-be74-e34a0a71452b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:35:57.668978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f355ccde-caac-4e0f-be74-e34a0a71452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:57.763445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 669/1321] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:35:57.802871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fb2b3-f6f8-4b3b-a2e4-8a13f655e9c7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:35:57.804462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7f3fb2b3-f6f8-4b3b-a2e4-8a13f655e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:57.908924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fb2b3-f6f8-4b3b-a2e4-8a13f655e9c7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:35:57.986649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:35:57.987300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:35:57.987586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:35:57.987870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:35:57.989599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:35:57.997287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:35:58.098482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:35:58.099921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af29e4-7219-4db0-92f5-beabdaa1c83d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:35:58.117933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 653/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:01.011642 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a447b-2e04-4f67-908a-1ecc8a3a757a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:01.012234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a447b-2e04-4f67-908a-1ecc8a3a757a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:01.012234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a447b-2e04-4f67-908a-1ecc8a3a757a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:01.013604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 670/1323] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:36:01.018829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db05ff-efc6-4fb6-b531-74ec7158738f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:01.019278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db05ff-efc6-4fb6-b531-74ec7158738f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:01.019919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db05ff-efc6-4fb6-b531-74ec7158738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:01.085011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c4db05ff-efc6-4fb6-b531-74ec7158738f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:01.086216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4db05ff-efc6-4fb6-b531-74ec7158738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:01.144197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:01.185773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 654/1324] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:01.221693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a447b-2e04-4f67-908a-1ecc8a3a757a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:01.223891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f20a447b-2e04-4f67-908a-1ecc8a3a757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:01.329175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a447b-2e04-4f67-908a-1ecc8a3a757a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:01.345414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:01.412294 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:01.412997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:01.413284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:01.413563 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:01.415269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:01.527511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:01.528919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6261-97cc-4018-b6cb-c0112f6ec9c3 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:01.546846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 671/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:03.838231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5aedd-c891-414a-840e-5988ba633297 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:03.838800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5aedd-c891-414a-840e-5988ba633297 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:03.838800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5aedd-c891-414a-840e-5988ba633297 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:03.840283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 655/1326] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:03.845619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf98b5f-fc32-4048-8443-cddf0ccdea56 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:03.846024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf98b5f-fc32-4048-8443-cddf0ccdea56 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:03.846264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf98b5f-fc32-4048-8443-cddf0ccdea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:03.894574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:03.921853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3cf98b5f-fc32-4048-8443-cddf0ccdea56 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:03.923084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3cf98b5f-fc32-4048-8443-cddf0ccdea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:04.023631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 672/1327] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:04.059882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5aedd-c891-414a-840e-5988ba633297 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:04.061585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58d5aedd-c891-414a-840e-5988ba633297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:04.168482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5aedd-c891-414a-840e-5988ba633297 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:04.251550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:04.252314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:04.252612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:04.252891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:04.254569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:04.271907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:04.366151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:04.367602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6873-e83f-4d2c-ae87-28d540fadfb1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:04.385890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 656/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:06.668872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e6f0b-0554-4b1c-87b8-05acdb76fa50 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:06.669373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e6f0b-0554-4b1c-87b8-05acdb76fa50 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:06.669373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e6f0b-0554-4b1c-87b8-05acdb76fa50 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:06.670731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 673/1329] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:06.676738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d3b06-191a-47a8-8cfe-bf7cde538273 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:06.677156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d3b06-191a-47a8-8cfe-bf7cde538273 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:06.677310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d3b06-191a-47a8-8cfe-bf7cde538273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:06.747677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c20d3b06-191a-47a8-8cfe-bf7cde538273 None None] MFA Rules not processed for user `a2bb088e25f642dba7a1e5a300b90359`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:06.748925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c20d3b06-191a-47a8-8cfe-bf7cde538273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:06.814808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:06.849648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 657/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:06.887616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e6f0b-0554-4b1c-87b8-05acdb76fa50 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:06.889256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b2e6f0b-0554-4b1c-87b8-05acdb76fa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:07.006290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e6f0b-0554-4b1c-87b8-05acdb76fa50 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:07.026947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:07.086723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:07.087381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:07.087668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:07.087944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:07.089631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:07.201149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:07.202613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85d6b1-d9a0-4795-82bd-a5a750a84355 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:07.221867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 674/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:09.886006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e18d62-5a77-423d-b762-747c282e908a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:09.886525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e18d62-5a77-423d-b762-747c282e908a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:09.886648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e18d62-5a77-423d-b762-747c282e908a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:09.887949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 658/1332] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:09.893314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92090b-720e-45fb-80ef-2db662b9f30d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:09.893767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92090b-720e-45fb-80ef-2db662b9f30d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:09.893865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92090b-720e-45fb-80ef-2db662b9f30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:09.965664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:09.970144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5d92090b-720e-45fb-80ef-2db662b9f30d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:09.971568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5d92090b-720e-45fb-80ef-2db662b9f30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:10.071142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 675/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:10.107013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e18d62-5a77-423d-b762-747c282e908a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:10.108568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6e18d62-5a77-423d-b762-747c282e908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:10.216931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e18d62-5a77-423d-b762-747c282e908a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:10.291713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:10.292388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:10.292672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:10.292951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:10.294590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:10.323347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:10.416045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:10.417619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de14e0-6df3-47e4-8cae-783382febc7c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:10.436995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 659/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:12.815877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fb1c-6a79-4f02-869d-c5ce4d85fd4f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:12.816779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fb1c-6a79-4f02-869d-c5ce4d85fd4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:12.816779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fb1c-6a79-4f02-869d-c5ce4d85fd4f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:12.817875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 676/1335] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:12.823974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20207fd-b732-4242-a1a2-b0f1a18d8497 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:12.824553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20207fd-b732-4242-a1a2-b0f1a18d8497 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:12.824677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20207fd-b732-4242-a1a2-b0f1a18d8497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:12.908983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b20207fd-b732-4242-a1a2-b0f1a18d8497 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:12.911122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b20207fd-b732-4242-a1a2-b0f1a18d8497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:13.009219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:13.032752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 660/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:13.073053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fb1c-6a79-4f02-869d-c5ce4d85fd4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:13.075023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4fd1fb1c-6a79-4f02-869d-c5ce4d85fd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:13.187724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fb1c-6a79-4f02-869d-c5ce4d85fd4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:13.221157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:13.271961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:13.272697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:13.273049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:13.273346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:13.275091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:13.408230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:13.409752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddb7d4-934c-4ae5-81ff-3e438e1377bb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:13.427633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 677/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:16.032277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc89f9-5dd0-4da1-a61f-b46793e6229a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:16.032834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc89f9-5dd0-4da1-a61f-b46793e6229a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:16.032931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc89f9-5dd0-4da1-a61f-b46793e6229a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:16.034265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 661/1338] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:16.040052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cc841-810e-4aee-80c9-63b916478d4d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:16.040520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cc841-810e-4aee-80c9-63b916478d4d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:16.040673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cc841-810e-4aee-80c9-63b916478d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:16.105503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-934cc841-810e-4aee-80c9-63b916478d4d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:16.106712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-934cc841-810e-4aee-80c9-63b916478d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:16.115619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:16.203600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 678/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:16.239645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc89f9-5dd0-4da1-a61f-b46793e6229a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:16.241241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7dcc89f9-5dd0-4da1-a61f-b46793e6229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:16.347049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc89f9-5dd0-4da1-a61f-b46793e6229a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:16.426145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:16.426801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:16.427085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:16.427363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:16.429088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:16.466798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:16.542220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:16.543668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6c0a3-c0e3-422b-a5e0-4ec6c4d047fe service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:16.562759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 662/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:20.905856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233cbb7-37f2-478d-bef4-b889105830b0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:20.906363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233cbb7-37f2-478d-bef4-b889105830b0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:20.906363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233cbb7-37f2-478d-bef4-b889105830b0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:20.907759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 679/1341] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:20.913726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d90e-ff37-4495-91ff-051c534b9a3f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:20.914206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d90e-ff37-4495-91ff-051c534b9a3f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:20.914206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d90e-ff37-4495-91ff-051c534b9a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:20.986651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2992d90e-ff37-4495-91ff-051c534b9a3f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:20.988681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2992d90e-ff37-4495-91ff-051c534b9a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:21.082694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:21.096958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 663/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:21.140438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233cbb7-37f2-478d-bef4-b889105830b0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:21.142562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4233cbb7-37f2-478d-bef4-b889105830b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:21.267672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233cbb7-37f2-478d-bef4-b889105830b0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:21.271608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ec1ce-4308-4ed4-95e5-7a88dc35b565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:21.272265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec1ce-4308-4ed4-95e5-7a88dc35b565 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:21.272573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec1ce-4308-4ed4-95e5-7a88dc35b565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:21.272848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec1ce-4308-4ed4-95e5-7a88dc35b565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:21.282217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ec1ce-4308-4ed4-95e5-7a88dc35b565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:21.283660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ec1ce-4308-4ed4-95e5-7a88dc35b565 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:21.293707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 680/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:24.216655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ba308-c259-48df-918d-fe44095121e0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:24.217190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ba308-c259-48df-918d-fe44095121e0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:24.217190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ba308-c259-48df-918d-fe44095121e0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:24.218770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 664/1344] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:24.224804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f299c12-e5a1-4bb0-8632-08320ed34f1b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:24.225248 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f299c12-e5a1-4bb0-8632-08320ed34f1b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:24.225517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f299c12-e5a1-4bb0-8632-08320ed34f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:24.256252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8f299c12-e5a1-4bb0-8632-08320ed34f1b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:24.257550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8f299c12-e5a1-4bb0-8632-08320ed34f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:24.286829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:24.343827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 681/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:36:24.381929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ba308-c259-48df-918d-fe44095121e0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:24.392473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ba308-c259-48df-918d-fe44095121e0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:24.396202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ab4b8-740a-4bf0-a5e4-ccaf1664a263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:24.396871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ab4b8-740a-4bf0-a5e4-ccaf1664a263 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:24.397157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ab4b8-740a-4bf0-a5e4-ccaf1664a263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:24.397436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ab4b8-740a-4bf0-a5e4-ccaf1664a263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:24.406750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ab4b8-740a-4bf0-a5e4-ccaf1664a263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:24.408219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ab4b8-740a-4bf0-a5e4-ccaf1664a263 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:24.419023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 665/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:28.666706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822e2f-aca5-48da-b708-0760f830fce8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:28.667216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822e2f-aca5-48da-b708-0760f830fce8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:28.667216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822e2f-aca5-48da-b708-0760f830fce8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:28.668603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 682/1347] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:28.674919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e8cfa-a8e5-49cf-ac88-32f463f223a3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:28.676027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e8cfa-a8e5-49cf-ac88-32f463f223a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:28.676027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e8cfa-a8e5-49cf-ac88-32f463f223a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:28.695686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf6e8cfa-a8e5-49cf-ac88-32f463f223a3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:28.696957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf6e8cfa-a8e5-49cf-ac88-32f463f223a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:28.705314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 666/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:36:28.742963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822e2f-aca5-48da-b708-0760f830fce8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:28.755113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822e2f-aca5-48da-b708-0760f830fce8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:28.758618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc933b7-fbf7-4d8c-b480-4313e400e908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:28.759259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc933b7-fbf7-4d8c-b480-4313e400e908 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:28.759548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc933b7-fbf7-4d8c-b480-4313e400e908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:28.759814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc933b7-fbf7-4d8c-b480-4313e400e908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:28.769972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc933b7-fbf7-4d8c-b480-4313e400e908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:28.771412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc933b7-fbf7-4d8c-b480-4313e400e908 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:28.782678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 683/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:34.298150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a3642-5e71-49f3-9937-6d9e8add4bfd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:34.299102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a3642-5e71-49f3-9937-6d9e8add4bfd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:34.300204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a3642-5e71-49f3-9937-6d9e8add4bfd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:34.302072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 667/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:36:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:34.312256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ef80f-3f6b-414c-8af1-b9af50146ed9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:34.313129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ef80f-3f6b-414c-8af1-b9af50146ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:34.313630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ef80f-3f6b-414c-8af1-b9af50146ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:34.446354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9b3ef80f-3f6b-414c-8af1-b9af50146ed9 None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:34.448593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b3ef80f-3f6b-414c-8af1-b9af50146ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:34.544357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 684/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:36:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:38.689414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=13 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:39.185676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a680344-40e7-4861-9bb0-e90a7bc16682 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:39.186042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a680344-40e7-4861-9bb0-e90a7bc16682 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:39.186277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a680344-40e7-4861-9bb0-e90a7bc16682 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:39.187940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 668/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:39.194012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2ea1-d4cf-467c-a100-5d82b2cf4c91 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:39.194572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2ea1-d4cf-467c-a100-5d82b2cf4c91 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:39.194572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b2ea1-d4cf-467c-a100-5d82b2cf4c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:39.215655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-650b2ea1-d4cf-467c-a100-5d82b2cf4c91 None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:39.217358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-650b2ea1-d4cf-467c-a100-5d82b2cf4c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:39.227710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 685/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:36:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:40.935276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8ebe6-7ba8-4948-a349-7008cf1bb483 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:40.935791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8ebe6-7ba8-4948-a349-7008cf1bb483 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:40.935791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8ebe6-7ba8-4948-a349-7008cf1bb483 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:40.937245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 669/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:40.943213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e2232-d8aa-4d40-b79c-851a2324534e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:40.943749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e2232-d8aa-4d40-b79c-851a2324534e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:40.943853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e2232-d8aa-4d40-b79c-851a2324534e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:40.964500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fb7e2232-d8aa-4d40-b79c-851a2324534e None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:40.965716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb7e2232-d8aa-4d40-b79c-851a2324534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:40.975244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 686/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:45.013165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e09ed-7e98-405b-b296-fd42121789a8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:45.013792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e09ed-7e98-405b-b296-fd42121789a8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:45.013792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e09ed-7e98-405b-b296-fd42121789a8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:45.016911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 670/1356] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:45.021508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1a6c5-3b91-429a-ba57-f7ada96e6761 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:45.022087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1a6c5-3b91-429a-ba57-f7ada96e6761 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:45.022087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1a6c5-3b91-429a-ba57-f7ada96e6761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:45.041533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ce1a6c5-3b91-429a-ba57-f7ada96e6761 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:45.042738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ce1a6c5-3b91-429a-ba57-f7ada96e6761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:45.051524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 687/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:36:45.118857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edaeb4-1162-4669-ac2d-33a488acf9a4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:45.119852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edaeb4-1162-4669-ac2d-33a488acf9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:45.121616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edaeb4-1162-4669-ac2d-33a488acf9a4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:45.124978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 671/1358] 10.222.0.16 () {54 vars in 961 bytes} [Thu May 26 13:36:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:45.137277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff492f-1829-44e0-b396-208ca4e086e9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:45.137553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff492f-1829-44e0-b396-208ca4e086e9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:45.137907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff492f-1829-44e0-b396-208ca4e086e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:45.189963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:45.199979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c6ff492f-1829-44e0-b396-208ca4e086e9 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:45.201469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6ff492f-1829-44e0-b396-208ca4e086e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:45.302251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 688/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Thu May 26 13:36:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:45.310167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edaeb4-1162-4669-ac2d-33a488acf9a4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:45.327294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edaeb4-1162-4669-ac2d-33a488acf9a4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:45.333798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe961-f4ad-48b3-a230-b13559a9836a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:45.334791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe961-f4ad-48b3-a230-b13559a9836a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:45.335667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe961-f4ad-48b3-a230-b13559a9836a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:45.336745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe961-f4ad-48b3-a230-b13559a9836a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:45.348028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fe961-f4ad-48b3-a230-b13559a9836a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:45.350038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fe961-f4ad-48b3-a230-b13559a9836a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:45.364115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 672/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:47.891198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c868cac-f086-4b72-a4b5-8c16218f894d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:47.891728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c868cac-f086-4b72-a4b5-8c16218f894d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:47.891811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c868cac-f086-4b72-a4b5-8c16218f894d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:47.893455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 689/1361] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:47.899125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699aa1f-676b-4f53-8bd1-f03484e2cd8c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:47.899593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699aa1f-676b-4f53-8bd1-f03484e2cd8c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:47.899677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699aa1f-676b-4f53-8bd1-f03484e2cd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:47.918553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f699aa1f-676b-4f53-8bd1-f03484e2cd8c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:47.920059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f699aa1f-676b-4f53-8bd1-f03484e2cd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:47.929292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 673/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:47.974283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c868cac-f086-4b72-a4b5-8c16218f894d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:47.990063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c868cac-f086-4b72-a4b5-8c16218f894d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:47.995792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cab488-b9d9-4557-a9c2-572d5d971e81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:47.996412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cab488-b9d9-4557-a9c2-572d5d971e81 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:47.996698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cab488-b9d9-4557-a9c2-572d5d971e81 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:47.996976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cab488-b9d9-4557-a9c2-572d5d971e81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:48.006735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cab488-b9d9-4557-a9c2-572d5d971e81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:48.008382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cab488-b9d9-4557-a9c2-572d5d971e81 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:48.019570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 690/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:50.633179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270787-91f8-4e88-9784-2e9d17ea56de None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:50.633698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270787-91f8-4e88-9784-2e9d17ea56de None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:50.633779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17270787-91f8-4e88-9784-2e9d17ea56de None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:50.635138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 674/1364] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:50.640973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9bb5-3e12-458b-8da4-ff039cce718a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:50.641494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9bb5-3e12-458b-8da4-ff039cce718a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:50.641589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9bb5-3e12-458b-8da4-ff039cce718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:50.661420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-84bc9bb5-3e12-458b-8da4-ff039cce718a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:50.662659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84bc9bb5-3e12-458b-8da4-ff039cce718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:50.671580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 691/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:50.722082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f7dd-2ff0-46f9-8400-b02ef678e96a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:50.722339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f7dd-2ff0-46f9-8400-b02ef678e96a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:50.722521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50f7dd-2ff0-46f9-8400-b02ef678e96a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:50.724022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 675/1366] 10.222.0.16 () {54 vars in 961 bytes} [Thu May 26 13:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:36:50.732680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da9a3-d97f-4b87-918c-059b2a797774 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:50.733092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da9a3-d97f-4b87-918c-059b2a797774 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:50.733377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da9a3-d97f-4b87-918c-059b2a797774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:50.755809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fa5da9a3-d97f-4b87-918c-059b2a797774 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:50.757363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa5da9a3-d97f-4b87-918c-059b2a797774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:50.768016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 692/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Thu May 26 13:36:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:50.776603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f7dd-2ff0-46f9-8400-b02ef678e96a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:50.792554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50f7dd-2ff0-46f9-8400-b02ef678e96a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:50.798463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350416a6-bf7f-43be-b074-d4f25ab694c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:50.799138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350416a6-bf7f-43be-b074-d4f25ab694c5 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:50.799420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350416a6-bf7f-43be-b074-d4f25ab694c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:50.799700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350416a6-bf7f-43be-b074-d4f25ab694c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:50.809288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350416a6-bf7f-43be-b074-d4f25ab694c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:50.810929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350416a6-bf7f-43be-b074-d4f25ab694c5 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:50.822579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 676/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:53.170397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69b95b-568f-446a-986e-01a311b6fa26 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:53.170995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69b95b-568f-446a-986e-01a311b6fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:53.170995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69b95b-568f-446a-986e-01a311b6fa26 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:53.172500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 693/1369] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:53.178338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b49e60-6067-419b-82dd-52d750452229 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:53.178803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b49e60-6067-419b-82dd-52d750452229 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:53.178915 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b49e60-6067-419b-82dd-52d750452229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:53.198156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4b49e60-6067-419b-82dd-52d750452229 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:53.199365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4b49e60-6067-419b-82dd-52d750452229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:53.208903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 677/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:53.255191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69b95b-568f-446a-986e-01a311b6fa26 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:53.270596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69b95b-568f-446a-986e-01a311b6fa26 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:53.277629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33e56-ab8a-49e6-94a0-3b862c50a96d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:53.278423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33e56-ab8a-49e6-94a0-3b862c50a96d service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:53.278704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33e56-ab8a-49e6-94a0-3b862c50a96d service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:53.279185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33e56-ab8a-49e6-94a0-3b862c50a96d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:53.289966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33e56-ab8a-49e6-94a0-3b862c50a96d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:53.291737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33e56-ab8a-49e6-94a0-3b862c50a96d service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:53.303142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 694/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:53.915804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50ef8-6e5b-4739-8e2f-35b4b31391aa None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:53.916322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50ef8-6e5b-4739-8e2f-35b4b31391aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:53.916431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c50ef8-6e5b-4739-8e2f-35b4b31391aa None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:53.918269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 678/1372] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:53.926277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50a6f5-512b-4bc0-b674-4406cb0bb6b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:53.926589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50a6f5-512b-4bc0-b674-4406cb0bb6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:53.926841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50a6f5-512b-4bc0-b674-4406cb0bb6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:53.948031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7f50a6f5-512b-4bc0-b674-4406cb0bb6b9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:53.949559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f50a6f5-512b-4bc0-b674-4406cb0bb6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:53.959155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 695/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:36:54.268541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353711-32a2-4c78-a183-d40db16575d5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:54.269092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353711-32a2-4c78-a183-d40db16575d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:54.269959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353711-32a2-4c78-a183-d40db16575d5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:54.271474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 679/1374] 10.222.0.16 () {54 vars in 963 bytes} [Thu May 26 13:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:54.279267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a563371-9545-4b79-ad58-8524bc1a66dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:54.279989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a563371-9545-4b79-ad58-8524bc1a66dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:54.279989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a563371-9545-4b79-ad58-8524bc1a66dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:54.335044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9a563371-9545-4b79-ad58-8524bc1a66dd None None] MFA Rules not processed for user `867e9b57fa7947e3afcbd974ed887969`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:54.336437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a563371-9545-4b79-ad58-8524bc1a66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:54.402793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 696/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Thu May 26 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:54.410876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d353711-32a2-4c78-a183-d40db16575d5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:54.422056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d353711-32a2-4c78-a183-d40db16575d5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:54.426084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f39572-be96-4f0a-962f-634b9447f0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:54.426846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39572-be96-4f0a-962f-634b9447f0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:54.427235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39572-be96-4f0a-962f-634b9447f0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:54.427636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39572-be96-4f0a-962f-634b9447f0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:54.439204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f39572-be96-4f0a-962f-634b9447f0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:54.441079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f39572-be96-4f0a-962f-634b9447f0f1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:54.451848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 680/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:55.641019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdda38-3417-44f8-977e-48cb8429ac89 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:55.641019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdda38-3417-44f8-977e-48cb8429ac89 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:55.641692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdda38-3417-44f8-977e-48cb8429ac89 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:55.643154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 697/1377] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:55.648738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c868bc1-9b59-4e45-bd5f-0108410a8b1c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:55.649181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c868bc1-9b59-4e45-bd5f-0108410a8b1c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:55.649449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c868bc1-9b59-4e45-bd5f-0108410a8b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:55.668175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c868bc1-9b59-4e45-bd5f-0108410a8b1c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:55.669427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c868bc1-9b59-4e45-bd5f-0108410a8b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:55.677381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 681/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:55.722316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdda38-3417-44f8-977e-48cb8429ac89 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:55.736545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdda38-3417-44f8-977e-48cb8429ac89 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:55.742043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb7fe5-92b1-4b6d-8558-eeb92faaa514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:55.742693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb7fe5-92b1-4b6d-8558-eeb92faaa514 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:55.742975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb7fe5-92b1-4b6d-8558-eeb92faaa514 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:55.743253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb7fe5-92b1-4b6d-8558-eeb92faaa514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:55.752444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb7fe5-92b1-4b6d-8558-eeb92faaa514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:55.754074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb7fe5-92b1-4b6d-8558-eeb92faaa514 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:55.758260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:36:55.765034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 698/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:36:58.090438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f32a0e-359c-4e41-896e-9064d7014790 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:58.090967 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f32a0e-359c-4e41-896e-9064d7014790 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:58.091062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f32a0e-359c-4e41-896e-9064d7014790 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:58.092452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 682/1380] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:36:58.098496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c6d8-b9cf-44d1-ad3d-641ed3353bfd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:58.098982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c6d8-b9cf-44d1-ad3d-641ed3353bfd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:58.099080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c6d8-b9cf-44d1-ad3d-641ed3353bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:58.117329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d109c6d8-b9cf-44d1-ad3d-641ed3353bfd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:36:58.118529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d109c6d8-b9cf-44d1-ad3d-641ed3353bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:36:58.138778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 699/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:36:58.187872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f32a0e-359c-4e41-896e-9064d7014790 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:36:58.210262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f32a0e-359c-4e41-896e-9064d7014790 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:36:58.218350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807f38e-6513-4e12-bf12-bc842ad14ac3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:36:58.219137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f38e-6513-4e12-bf12-bc842ad14ac3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:36:58.219531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f38e-6513-4e12-bf12-bc842ad14ac3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:36:58.220826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f38e-6513-4e12-bf12-bc842ad14ac3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:36:58.233198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807f38e-6513-4e12-bf12-bc842ad14ac3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:36:58.235752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807f38e-6513-4e12-bf12-bc842ad14ac3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:36:58.248162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 683/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:00.573166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc0b09-0fd5-48ee-8940-95a6acf7f7a3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:00.573771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc0b09-0fd5-48ee-8940-95a6acf7f7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:00.573771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc0b09-0fd5-48ee-8940-95a6acf7f7a3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:00.575107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 700/1383] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:00.581127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d510f-7ef6-4eb7-bd8a-f526300aa10e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:00.581588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d510f-7ef6-4eb7-bd8a-f526300aa10e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:00.581888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d510f-7ef6-4eb7-bd8a-f526300aa10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:00.601496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-351d510f-7ef6-4eb7-bd8a-f526300aa10e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:00.602685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-351d510f-7ef6-4eb7-bd8a-f526300aa10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:00.611786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 684/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:00.653178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc0b09-0fd5-48ee-8940-95a6acf7f7a3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:00.668713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc0b09-0fd5-48ee-8940-95a6acf7f7a3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:00.675595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09b9e4-08a8-4384-a186-97a2e186d6c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:00.676608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09b9e4-08a8-4384-a186-97a2e186d6c7 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:00.677117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09b9e4-08a8-4384-a186-97a2e186d6c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:00.677591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09b9e4-08a8-4384-a186-97a2e186d6c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:00.686966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09b9e4-08a8-4384-a186-97a2e186d6c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:00.688946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09b9e4-08a8-4384-a186-97a2e186d6c7 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:00.707075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 701/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:03.046188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fa53-749c-4f12-829f-11d389e64778 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:03.046730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fa53-749c-4f12-829f-11d389e64778 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:03.046730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4fa53-749c-4f12-829f-11d389e64778 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:03.049394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 685/1386] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:37:03.055268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a33a4-8797-4a8e-b4f4-89eb4de4712e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:03.055970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a33a4-8797-4a8e-b4f4-89eb4de4712e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:03.055970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a33a4-8797-4a8e-b4f4-89eb4de4712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:03.089684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8b1a33a4-8797-4a8e-b4f4-89eb4de4712e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:03.090965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8b1a33a4-8797-4a8e-b4f4-89eb4de4712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:03.111028 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 702/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:37:03.159905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fa53-749c-4f12-829f-11d389e64778 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:03.179254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4fa53-749c-4f12-829f-11d389e64778 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:03.186724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e10ec6-32f6-4337-8bbb-77d87e77cefe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:03.187426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10ec6-32f6-4337-8bbb-77d87e77cefe service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:03.187779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10ec6-32f6-4337-8bbb-77d87e77cefe service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:03.188326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e10ec6-32f6-4337-8bbb-77d87e77cefe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:03.201167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e10ec6-32f6-4337-8bbb-77d87e77cefe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:03.203247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e10ec6-32f6-4337-8bbb-77d87e77cefe service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:03.217824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 686/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:05.865944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae5aaf-8723-496d-8756-3c8483545961 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:05.866510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae5aaf-8723-496d-8756-3c8483545961 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:05.866510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae5aaf-8723-496d-8756-3c8483545961 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:05.867878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 703/1389] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:05.873730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17608-50ec-469b-93af-81be5ca89611 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:05.874499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17608-50ec-469b-93af-81be5ca89611 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:05.874499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17608-50ec-469b-93af-81be5ca89611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:05.893393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0fa17608-50ec-469b-93af-81be5ca89611 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:05.894653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0fa17608-50ec-469b-93af-81be5ca89611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:05.902927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 687/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:05.951405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae5aaf-8723-496d-8756-3c8483545961 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:05.967555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae5aaf-8723-496d-8756-3c8483545961 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:05.975240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e3152-ef81-4506-a851-7e4ba0fe9098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:05.976106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e3152-ef81-4506-a851-7e4ba0fe9098 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:05.976571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e3152-ef81-4506-a851-7e4ba0fe9098 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:05.976862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e3152-ef81-4506-a851-7e4ba0fe9098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:05.986782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e3152-ef81-4506-a851-7e4ba0fe9098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:05.988422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e3152-ef81-4506-a851-7e4ba0fe9098 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:05.998293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 704/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:08.337096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f0a7d-697c-4900-82ce-21f00811043c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:08.337663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f0a7d-697c-4900-82ce-21f00811043c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:08.337663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f0a7d-697c-4900-82ce-21f00811043c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:08.339107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 688/1392] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:08.345114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763d0b1-3cd4-41e3-8a05-617a9f1028dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:08.345640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763d0b1-3cd4-41e3-8a05-617a9f1028dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:08.345762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763d0b1-3cd4-41e3-8a05-617a9f1028dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:08.364553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e763d0b1-3cd4-41e3-8a05-617a9f1028dd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:08.365786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e763d0b1-3cd4-41e3-8a05-617a9f1028dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:08.375691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 705/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:08.430071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f0a7d-697c-4900-82ce-21f00811043c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:08.444391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f0a7d-697c-4900-82ce-21f00811043c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:08.450232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb654d5-1823-4399-8cb4-f01fb31224af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:08.450871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb654d5-1823-4399-8cb4-f01fb31224af service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:08.451151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb654d5-1823-4399-8cb4-f01fb31224af service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:08.451444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb654d5-1823-4399-8cb4-f01fb31224af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:08.461958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb654d5-1823-4399-8cb4-f01fb31224af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:08.463594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb654d5-1823-4399-8cb4-f01fb31224af service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:08.473799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 689/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:10.647551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c7d39-a178-4eca-b3ca-91919f16f695 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.648139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c7d39-a178-4eca-b3ca-91919f16f695 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.648139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c7d39-a178-4eca-b3ca-91919f16f695 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.649679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 706/1395] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:10.654932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2bbda-a7d2-4c63-b10b-4ed7222cfa07 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.655427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2bbda-a7d2-4c63-b10b-4ed7222cfa07 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.655528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2bbda-a7d2-4c63-b10b-4ed7222cfa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.662190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=52 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:37:10.677482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-14a2bbda-a7d2-4c63-b10b-4ed7222cfa07 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:10.678759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-14a2bbda-a7d2-4c63-b10b-4ed7222cfa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:10.686694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 690/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:10.744267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641347b-fb06-40e8-8291-8b4385bcdad6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.744645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641347b-fb06-40e8-8291-8b4385bcdad6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.744909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641347b-fb06-40e8-8291-8b4385bcdad6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.746310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 707/1397] 10.222.0.16 () {54 vars in 962 bytes} [Thu May 26 13:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:10.751821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.752203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.752645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.803378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] MFA Rules not processed for user `4e8552f561ce4b4fa837f907a58816aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:10.804609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:10.815883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b4e1a-d09c-4edb-b3f2-fc558148beca None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.816174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b4e1a-d09c-4edb-b3f2-fc558148beca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.816496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b4e1a-d09c-4edb-b3f2-fc558148beca None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.818097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 708/1398] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:37:10.823300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.823599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.823882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.842545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:10.844094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:10.852287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 709/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:10.866513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 691/1400] 10.222.0.16 () {58 vars in 1059 bytes} [Thu May 26 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:10.874625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:10.885127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d845ab-f746-45e4-9999-19909be7cc2e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:10.888813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f90363-8361-44a3-a634-79b8d95e4ad6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:10.889471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90363-8361-44a3-a634-79b8d95e4ad6 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.889754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90363-8361-44a3-a634-79b8d95e4ad6 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.890031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90363-8361-44a3-a634-79b8d95e4ad6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.900099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f90363-8361-44a3-a634-79b8d95e4ad6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:10.900963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:10.901583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f90363-8361-44a3-a634-79b8d95e4ad6 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:10.912446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 710/1401] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:10.914596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d60b8-ab05-47a2-a087-05228cfd4673 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:10.920263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9adb30-95a7-4823-9c54-96924ec3ed6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:10.920957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9adb30-95a7-4823-9c54-96924ec3ed6a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:10.921243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9adb30-95a7-4823-9c54-96924ec3ed6a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:10.921556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9adb30-95a7-4823-9c54-96924ec3ed6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:10.931057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9adb30-95a7-4823-9c54-96924ec3ed6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:10.932708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9adb30-95a7-4823-9c54-96924ec3ed6a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:10.942894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 692/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:37:11.468228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5c342-3f7c-4abd-bd9e-34a5757bb539 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:11.468559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5c342-3f7c-4abd-bd9e-34a5757bb539 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:11.468842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5c342-3f7c-4abd-bd9e-34a5757bb539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:11.521570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e4f5c342-3f7c-4abd-bd9e-34a5757bb539 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:11.522873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4f5c342-3f7c-4abd-bd9e-34a5757bb539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:11.590279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 711/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:37:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:11.635188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b23fd-5645-4228-9f56-ff80fe5c1c16 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:11.635481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b23fd-5645-4228-9f56-ff80fe5c1c16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:11.635760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b23fd-5645-4228-9f56-ff80fe5c1c16 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:11.637183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 693/1404] 10.222.0.16 () {54 vars in 942 bytes} [Thu May 26 13:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:11.643510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c65b-65c7-4310-8806-d8b765e5a1df None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:11.643808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c65b-65c7-4310-8806-d8b765e5a1df None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:11.644121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c65b-65c7-4310-8806-d8b765e5a1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:11.695076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fda3c65b-65c7-4310-8806-d8b765e5a1df None None] MFA Rules not processed for user `e4e2c8a980714257b8df1d486d446eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:11.696286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fda3c65b-65c7-4310-8806-d8b765e5a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:11.732228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:37:11.773561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 712/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Thu May 26 13:37:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:11.783191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b23fd-5645-4228-9f56-ff80fe5c1c16 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:11.796695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b23fd-5645-4228-9f56-ff80fe5c1c16 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:11.802439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d63b1-32cd-4b90-931e-4db555b63f3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:11.803082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d63b1-32cd-4b90-931e-4db555b63f3c service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:11.803363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d63b1-32cd-4b90-931e-4db555b63f3c service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:11.803641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d63b1-32cd-4b90-931e-4db555b63f3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:11.812120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d63b1-32cd-4b90-931e-4db555b63f3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:11.814404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d63b1-32cd-4b90-931e-4db555b63f3c service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:11.823082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 694/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:37:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:37:13.298653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ae44-f423-43c3-a1ae-2654a54cebac None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:13.299250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ae44-f423-43c3-a1ae-2654a54cebac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:13.299250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ae44-f423-43c3-a1ae-2654a54cebac None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:13.300835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 713/1407] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:37:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:13.308013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394ae4a-9b15-42e9-b788-cddaba78af01 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:13.308591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394ae4a-9b15-42e9-b788-cddaba78af01 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:13.308591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394ae4a-9b15-42e9-b788-cddaba78af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:13.330756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0394ae4a-9b15-42e9-b788-cddaba78af01 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:13.332045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0394ae4a-9b15-42e9-b788-cddaba78af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:13.341613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 695/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:13.392561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469ae44-f423-43c3-a1ae-2654a54cebac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:13.408767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469ae44-f423-43c3-a1ae-2654a54cebac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:13.415344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd0a97-ac07-454a-a18d-cfbf6930c43f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:13.416003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd0a97-ac07-454a-a18d-cfbf6930c43f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:13.416290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd0a97-ac07-454a-a18d-cfbf6930c43f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:13.416587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd0a97-ac07-454a-a18d-cfbf6930c43f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:13.430881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd0a97-ac07-454a-a18d-cfbf6930c43f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:13.430881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd0a97-ac07-454a-a18d-cfbf6930c43f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:13.443307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 714/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:37:23.322563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:37:23.436671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=21 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:37:35.804626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616337-0166-4f0e-91c6-93f8cbac3517 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:35.805252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616337-0166-4f0e-91c6-93f8cbac3517 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:35.805252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616337-0166-4f0e-91c6-93f8cbac3517 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:35.806574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 696/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu May 26 13:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:35.812458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acbcb8-6f34-4052-b094-b30632fc6057 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:35.813000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acbcb8-6f34-4052-b094-b30632fc6057 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:35.813000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acbcb8-6f34-4052-b094-b30632fc6057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:35.831200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-65acbcb8-6f34-4052-b094-b30632fc6057 None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:35.832444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-65acbcb8-6f34-4052-b094-b30632fc6057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:35.840923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 715/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu May 26 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:35.855672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f603eb1-0400-478b-892e-208556a13a51 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:35.855959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f603eb1-0400-478b-892e-208556a13a51 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:35.856239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f603eb1-0400-478b-892e-208556a13a51 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:35.857656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 697/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu May 26 13:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:35.863347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9dec3-408a-42cf-81a3-011bb508d553 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:35.863668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9dec3-408a-42cf-81a3-011bb508d553 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:35.863981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9dec3-408a-42cf-81a3-011bb508d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:35.882299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-77f9dec3-408a-42cf-81a3-011bb508d553 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:35.883510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77f9dec3-408a-42cf-81a3-011bb508d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:35.892751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 716/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu May 26 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:38.636280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd325079-298f-466c-a741-5d7dc82b104f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:38.637041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd325079-298f-466c-a741-5d7dc82b104f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:38.637041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd325079-298f-466c-a741-5d7dc82b104f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:38.639415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:38.645352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8b896-26d7-40a6-804f-966928d673f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:38.645804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8b896-26d7-40a6-804f-966928d673f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:38.645910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8b896-26d7-40a6-804f-966928d673f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:38.664672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-63d8b896-26d7-40a6-804f-966928d673f8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:38.665942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63d8b896-26d7-40a6-804f-966928d673f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:38.674637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:38.717409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd325079-298f-466c-a741-5d7dc82b104f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:38.732165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd325079-298f-466c-a741-5d7dc82b104f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:38.737851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c9c2d-f8a2-4dc4-99b0-fe8e0e953d50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:38.738496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9c2d-f8a2-4dc4-99b0-fe8e0e953d50 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:38.738781 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9c2d-f8a2-4dc4-99b0-fe8e0e953d50 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:38.739062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c9c2d-f8a2-4dc4-99b0-fe8e0e953d50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:38.747415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9c2d-f8a2-4dc4-99b0-fe8e0e953d50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:38.749188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c9c2d-f8a2-4dc4-99b0-fe8e0e953d50 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:38.759657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 699/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:39.745963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ed82-dc8b-46ee-8e66-fe1a052d6e54 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:39.746534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ed82-dc8b-46ee-8e66-fe1a052d6e54 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:39.746534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ed82-dc8b-46ee-8e66-fe1a052d6e54 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:39.748113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:39.753906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4db06b-8c77-4a25-b370-4210d7c1acf5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:39.754399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4db06b-8c77-4a25-b370-4210d7c1acf5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:39.754497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4db06b-8c77-4a25-b370-4210d7c1acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:39.773140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d4db06b-8c77-4a25-b370-4210d7c1acf5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:39.774351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d4db06b-8c77-4a25-b370-4210d7c1acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:39.783073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:39.949911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ed82-dc8b-46ee-8e66-fe1a052d6e54 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:39.964999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ed82-dc8b-46ee-8e66-fe1a052d6e54 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:39.970435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3ca7f-5e7b-4b12-b786-9332c4e86fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:39.971074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3ca7f-5e7b-4b12-b786-9332c4e86fac service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:39.971391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3ca7f-5e7b-4b12-b786-9332c4e86fac service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:39.971676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3ca7f-5e7b-4b12-b786-9332c4e86fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:39.979815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3ca7f-5e7b-4b12-b786-9332c4e86fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:39.981439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3ca7f-5e7b-4b12-b786-9332c4e86fac service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:39.992100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 719/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:40.943197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd5133-ecd4-418a-a658-a49889bc574a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:40.943712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd5133-ecd4-418a-a658-a49889bc574a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:40.943810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd5133-ecd4-418a-a658-a49889bc574a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:40.945140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:40.950240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46962e3c-3bea-4f3e-b4cf-8d6f2510a34b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:40.950686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46962e3c-3bea-4f3e-b4cf-8d6f2510a34b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:40.950786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46962e3c-3bea-4f3e-b4cf-8d6f2510a34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:40.970367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-46962e3c-3bea-4f3e-b4cf-8d6f2510a34b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:40.971565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46962e3c-3bea-4f3e-b4cf-8d6f2510a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:40.979272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:37:41.130839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd5133-ecd4-418a-a658-a49889bc574a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:41.146120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd5133-ecd4-418a-a658-a49889bc574a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:41.151428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b88d0-f212-420d-85b7-65f7093e29a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:41.152071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b88d0-f212-420d-85b7-65f7093e29a1 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:41.152383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b88d0-f212-420d-85b7-65f7093e29a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:41.152658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b88d0-f212-420d-85b7-65f7093e29a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:41.161498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b88d0-f212-420d-85b7-65f7093e29a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:41.163087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b88d0-f212-420d-85b7-65f7093e29a1 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:41.173189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 702/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:42.097678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14be12f-16cf-4a89-b8e5-4437f9a249b2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:42.098270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14be12f-16cf-4a89-b8e5-4437f9a249b2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:42.098270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14be12f-16cf-4a89-b8e5-4437f9a249b2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:42.099630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:42.104678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff9711-dbd8-437a-91ba-ea83b1ff1a96 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:42.105107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff9711-dbd8-437a-91ba-ea83b1ff1a96 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:42.105278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff9711-dbd8-437a-91ba-ea83b1ff1a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:42.123811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0ff9711-dbd8-437a-91ba-ea83b1ff1a96 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:42.125048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0ff9711-dbd8-437a-91ba-ea83b1ff1a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:42.133068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:42.284846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14be12f-16cf-4a89-b8e5-4437f9a249b2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:42.299595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14be12f-16cf-4a89-b8e5-4437f9a249b2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:42.305308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6162e1-16f7-42f3-9e48-af9696a161c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:42.305954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6162e1-16f7-42f3-9e48-af9696a161c8 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:42.306239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6162e1-16f7-42f3-9e48-af9696a161c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:42.306515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6162e1-16f7-42f3-9e48-af9696a161c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:42.315875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6162e1-16f7-42f3-9e48-af9696a161c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:42.317575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6162e1-16f7-42f3-9e48-af9696a161c8 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:42.327799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 722/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:43.239954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4da59-66d3-416d-875b-4943a570ea31 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:43.240524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4da59-66d3-416d-875b-4943a570ea31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:43.240635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4da59-66d3-416d-875b-4943a570ea31 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:43.241937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:43.246962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d570c-4ef2-4eaf-a691-a9718b3146f5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:43.247385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d570c-4ef2-4eaf-a691-a9718b3146f5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:43.247524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d570c-4ef2-4eaf-a691-a9718b3146f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:43.265942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-354d570c-4ef2-4eaf-a691-a9718b3146f5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:43.267151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-354d570c-4ef2-4eaf-a691-a9718b3146f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:43.275038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:43.423411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4da59-66d3-416d-875b-4943a570ea31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:43.444100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4da59-66d3-416d-875b-4943a570ea31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:43.449649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4045f4-3f3a-4568-933e-3996fb4e16ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:43.450230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4045f4-3f3a-4568-933e-3996fb4e16ba service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:43.450514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4045f4-3f3a-4568-933e-3996fb4e16ba service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:43.450792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4045f4-3f3a-4568-933e-3996fb4e16ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:43.459827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4045f4-3f3a-4568-933e-3996fb4e16ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:43.461447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4045f4-3f3a-4568-933e-3996fb4e16ba service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:43.470868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 705/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:44.347770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6d02f-24b7-4e41-8e60-4399444cb03c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:44.348360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6d02f-24b7-4e41-8e60-4399444cb03c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:44.348360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6d02f-24b7-4e41-8e60-4399444cb03c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:44.349919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:44.354991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334af1f-a4c3-4c19-bb79-66333a933729 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:44.355432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334af1f-a4c3-4c19-bb79-66333a933729 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:44.355562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334af1f-a4c3-4c19-bb79-66333a933729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:44.374765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b334af1f-a4c3-4c19-bb79-66333a933729 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:44.375984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b334af1f-a4c3-4c19-bb79-66333a933729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:44.384112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:44.537043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6d02f-24b7-4e41-8e60-4399444cb03c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:44.551507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6d02f-24b7-4e41-8e60-4399444cb03c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:44.556942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda531-ef53-430a-b3d0-aabf1bbd124f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:44.557580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda531-ef53-430a-b3d0-aabf1bbd124f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:44.557868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda531-ef53-430a-b3d0-aabf1bbd124f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:44.558149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda531-ef53-430a-b3d0-aabf1bbd124f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:44.566410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffda531-ef53-430a-b3d0-aabf1bbd124f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:44.567996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffda531-ef53-430a-b3d0-aabf1bbd124f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:44.578396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 725/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:45.430049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b576673-f4f7-4693-a2e1-ee4efb715e45 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:45.430618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b576673-f4f7-4693-a2e1-ee4efb715e45 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:45.430618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b576673-f4f7-4693-a2e1-ee4efb715e45 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:45.432006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:37:45.437375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c17dc-2748-4a0e-8261-5617ffe8129e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:45.437854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c17dc-2748-4a0e-8261-5617ffe8129e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:45.437954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c17dc-2748-4a0e-8261-5617ffe8129e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:45.457336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-465c17dc-2748-4a0e-8261-5617ffe8129e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:45.458445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-465c17dc-2748-4a0e-8261-5617ffe8129e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:45.466164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:45.618288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b576673-f4f7-4693-a2e1-ee4efb715e45 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:45.633078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b576673-f4f7-4693-a2e1-ee4efb715e45 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:45.638409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e1808-3231-49a3-99b3-591b67b556c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:45.639047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e1808-3231-49a3-99b3-591b67b556c8 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:45.640197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e1808-3231-49a3-99b3-591b67b556c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:45.640522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e1808-3231-49a3-99b3-591b67b556c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:45.648754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e1808-3231-49a3-99b3-591b67b556c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:45.650346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e1808-3231-49a3-99b3-591b67b556c8 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:45.659725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 708/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:46.523963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2d68e-f9d3-4b55-8d84-cb579d284294 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:46.524515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2d68e-f9d3-4b55-8d84-cb579d284294 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:46.524975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2d68e-f9d3-4b55-8d84-cb579d284294 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:46.526381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:46.531589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712ff29-9db5-4c6d-bacf-a4727bd8f52a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:46.532064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712ff29-9db5-4c6d-bacf-a4727bd8f52a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:46.532150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712ff29-9db5-4c6d-bacf-a4727bd8f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:46.550732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6712ff29-9db5-4c6d-bacf-a4727bd8f52a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:46.551940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6712ff29-9db5-4c6d-bacf-a4727bd8f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:46.559760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:46.712208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2d68e-f9d3-4b55-8d84-cb579d284294 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:46.727062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2d68e-f9d3-4b55-8d84-cb579d284294 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:46.732230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b80280-94cb-4dd9-9425-1d5fbec5771f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:46.732891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b80280-94cb-4dd9-9425-1d5fbec5771f service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:46.733672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b80280-94cb-4dd9-9425-1d5fbec5771f service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:46.733942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b80280-94cb-4dd9-9425-1d5fbec5771f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:46.741912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b80280-94cb-4dd9-9425-1d5fbec5771f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:46.743489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b80280-94cb-4dd9-9425-1d5fbec5771f service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:46.753460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 728/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:47.612937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49fd49-2e67-46cb-a32d-dfd25c2e2d0c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:47.613514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49fd49-2e67-46cb-a32d-dfd25c2e2d0c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:47.613514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49fd49-2e67-46cb-a32d-dfd25c2e2d0c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:47.614918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:47.619934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1753f-d73e-4cd1-b3d8-7a87f92a9804 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:47.620443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1753f-d73e-4cd1-b3d8-7a87f92a9804 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:47.620574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1753f-d73e-4cd1-b3d8-7a87f92a9804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:47.639146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2fc1753f-d73e-4cd1-b3d8-7a87f92a9804 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:47.640574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fc1753f-d73e-4cd1-b3d8-7a87f92a9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:47.648373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:47.800492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49fd49-2e67-46cb-a32d-dfd25c2e2d0c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:47.815491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49fd49-2e67-46cb-a32d-dfd25c2e2d0c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:47.821229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c8e20-e225-448b-af28-2533b1aeb19c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:47.821878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8e20-e225-448b-af28-2533b1aeb19c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:47.822168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8e20-e225-448b-af28-2533b1aeb19c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:47.822451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8e20-e225-448b-af28-2533b1aeb19c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:47.831553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c8e20-e225-448b-af28-2533b1aeb19c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:47.833192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c8e20-e225-448b-af28-2533b1aeb19c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:47.843184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 711/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:49.962919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38bc27-3727-4c5c-8dfe-64ecc048185e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:49.963468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38bc27-3727-4c5c-8dfe-64ecc048185e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:49.963468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38bc27-3727-4c5c-8dfe-64ecc048185e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:49.964892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:49.971200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171b54a-e59b-4925-a523-6e7a47ad8485 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:49.971665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171b54a-e59b-4925-a523-6e7a47ad8485 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:49.971774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171b54a-e59b-4925-a523-6e7a47ad8485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:49.989869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a171b54a-e59b-4925-a523-6e7a47ad8485 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:49.991075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a171b54a-e59b-4925-a523-6e7a47ad8485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:49.999025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:50.037278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38bc27-3727-4c5c-8dfe-64ecc048185e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:50.048461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38bc27-3727-4c5c-8dfe-64ecc048185e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:50.052381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c448e0-f39f-4a19-8de4-81f4e8906618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:50.053023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c448e0-f39f-4a19-8de4-81f4e8906618 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:50.053313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c448e0-f39f-4a19-8de4-81f4e8906618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:50.053600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c448e0-f39f-4a19-8de4-81f4e8906618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:50.062772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c448e0-f39f-4a19-8de4-81f4e8906618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:50.064227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c448e0-f39f-4a19-8de4-81f4e8906618 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:50.073706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 731/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:52.502754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b05ce-ef62-4c1d-8621-79818de9e984 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:52.503635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b05ce-ef62-4c1d-8621-79818de9e984 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:52.503635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b05ce-ef62-4c1d-8621-79818de9e984 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:52.504849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:37:52.510539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc874b5b-8086-47f0-8d9f-44fd03d938b7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:52.511134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc874b5b-8086-47f0-8d9f-44fd03d938b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:52.511134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc874b5b-8086-47f0-8d9f-44fd03d938b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:52.529851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bc874b5b-8086-47f0-8d9f-44fd03d938b7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:52.530735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc874b5b-8086-47f0-8d9f-44fd03d938b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:52.538802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:52.578752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b05ce-ef62-4c1d-8621-79818de9e984 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:52.589903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b05ce-ef62-4c1d-8621-79818de9e984 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:52.593772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1025fd1-362a-4e00-9d5b-ec4ca8c74714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:52.594683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1025fd1-362a-4e00-9d5b-ec4ca8c74714 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:52.594963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1025fd1-362a-4e00-9d5b-ec4ca8c74714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:52.595445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1025fd1-362a-4e00-9d5b-ec4ca8c74714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:52.606291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1025fd1-362a-4e00-9d5b-ec4ca8c74714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:52.607810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1025fd1-362a-4e00-9d5b-ec4ca8c74714 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:52.621611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 714/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:55.202575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96319ba2-7d06-4e30-a7ec-774d7cb3b221 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:55.203032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96319ba2-7d06-4e30-a7ec-774d7cb3b221 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:55.203179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96319ba2-7d06-4e30-a7ec-774d7cb3b221 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:55.204866 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:55.210484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9aaeda-d3dc-4171-abce-76a2f4807738 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:55.210961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9aaeda-d3dc-4171-abce-76a2f4807738 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:55.211060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9aaeda-d3dc-4171-abce-76a2f4807738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:55.229435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d9aaeda-d3dc-4171-abce-76a2f4807738 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:55.230701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d9aaeda-d3dc-4171-abce-76a2f4807738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:55.238787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:55.277775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96319ba2-7d06-4e30-a7ec-774d7cb3b221 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:55.290226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96319ba2-7d06-4e30-a7ec-774d7cb3b221 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:55.293723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc796c-4974-4649-b7e3-1a719f8e5434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:55.294368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc796c-4974-4649-b7e3-1a719f8e5434 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:55.294653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc796c-4974-4649-b7e3-1a719f8e5434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:55.294933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc796c-4974-4649-b7e3-1a719f8e5434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:55.303807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc796c-4974-4649-b7e3-1a719f8e5434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:55.305275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc796c-4974-4649-b7e3-1a719f8e5434 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:55.314838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 734/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:37:57.669869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70b81f-e177-4a95-b558-395028cba825 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:57.670548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70b81f-e177-4a95-b558-395028cba825 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:57.670548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70b81f-e177-4a95-b558-395028cba825 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:57.671996 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:37:57.677349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9c16-3781-4cb4-91d2-28ae05130106 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:57.677787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9c16-3781-4cb4-91d2-28ae05130106 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:57.678011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9c16-3781-4cb4-91d2-28ae05130106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:57.696802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-562a9c16-3781-4cb4-91d2-28ae05130106 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:37:57.698012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-562a9c16-3781-4cb4-91d2-28ae05130106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:37:57.706753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:37:57.745477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70b81f-e177-4a95-b558-395028cba825 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:37:57.756843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70b81f-e177-4a95-b558-395028cba825 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:37:57.760367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc36e2-d147-4ba1-9590-2ef0dd5f1b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:37:57.761001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc36e2-d147-4ba1-9590-2ef0dd5f1b7b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:37:57.761286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc36e2-d147-4ba1-9590-2ef0dd5f1b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:37:57.761563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc36e2-d147-4ba1-9590-2ef0dd5f1b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:37:57.771702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc36e2-d147-4ba1-9590-2ef0dd5f1b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:37:57.773200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc36e2-d147-4ba1-9590-2ef0dd5f1b7b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:37:57.782785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 717/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:38:00.131474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53968e46-bb4a-4953-a169-c3c11948ebc2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:00.132059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53968e46-bb4a-4953-a169-c3c11948ebc2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:00.132059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53968e46-bb4a-4953-a169-c3c11948ebc2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:00.133514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:00.139531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc99d-2c08-4e91-882e-3de4a30b1ffd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:00.139945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc99d-2c08-4e91-882e-3de4a30b1ffd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:00.140109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc99d-2c08-4e91-882e-3de4a30b1ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:00.157961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-483dc99d-2c08-4e91-882e-3de4a30b1ffd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:00.159255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-483dc99d-2c08-4e91-882e-3de4a30b1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:00.167311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:00.206825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53968e46-bb4a-4953-a169-c3c11948ebc2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:00.218247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53968e46-bb4a-4953-a169-c3c11948ebc2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:00.221923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1da3d-35d6-4736-b97a-e3a498c7a1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:00.222617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1da3d-35d6-4736-b97a-e3a498c7a1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:00.222899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1da3d-35d6-4736-b97a-e3a498c7a1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:00.223183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1da3d-35d6-4736-b97a-e3a498c7a1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:00.233366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1da3d-35d6-4736-b97a-e3a498c7a1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:00.234848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1da3d-35d6-4736-b97a-e3a498c7a1c4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:00.245138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 737/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:02.504389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb1b90-5e79-4895-8053-87637a1e1c23 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:02.504895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb1b90-5e79-4895-8053-87637a1e1c23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:02.505001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb1b90-5e79-4895-8053-87637a1e1c23 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:02.506348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:02.511929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c704df-6c5a-40ac-b2fe-b43232354167 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:02.512431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c704df-6c5a-40ac-b2fe-b43232354167 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:02.512546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c704df-6c5a-40ac-b2fe-b43232354167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:02.529987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-50c704df-6c5a-40ac-b2fe-b43232354167 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:02.531196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50c704df-6c5a-40ac-b2fe-b43232354167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:02.538912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:02.578776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb1b90-5e79-4895-8053-87637a1e1c23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:02.589487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb1b90-5e79-4895-8053-87637a1e1c23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:02.593253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2494f-9d54-4d96-899c-6228edc68daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:02.593893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2494f-9d54-4d96-899c-6228edc68daf service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:02.594166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2494f-9d54-4d96-899c-6228edc68daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:02.594449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2494f-9d54-4d96-899c-6228edc68daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:02.603413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2494f-9d54-4d96-899c-6228edc68daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:02.604978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2494f-9d54-4d96-899c-6228edc68daf service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:02.615197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 720/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:04.869286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7632-c3c8-47d8-885a-24c293451fb1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:04.869859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7632-c3c8-47d8-885a-24c293451fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:04.869859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7632-c3c8-47d8-885a-24c293451fb1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:04.871201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:04.876730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4db5-ed0e-4504-a3c2-f86c4f55f412 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:04.877191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4db5-ed0e-4504-a3c2-f86c4f55f412 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:04.877297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4db5-ed0e-4504-a3c2-f86c4f55f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:04.883768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=70 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:38:04.897403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-862b4db5-ed0e-4504-a3c2-f86c4f55f412 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:04.898588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-862b4db5-ed0e-4504-a3c2-f86c4f55f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:04.907058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:04.943497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7632-c3c8-47d8-885a-24c293451fb1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:04.948968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=73 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:38:04.956380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7632-c3c8-47d8-885a-24c293451fb1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:04.959897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadf757-4615-4657-b68e-d440b1e076f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:04.960569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadf757-4615-4657-b68e-d440b1e076f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:04.960859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadf757-4615-4657-b68e-d440b1e076f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:04.961141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadf757-4615-4657-b68e-d440b1e076f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:04.971722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadf757-4615-4657-b68e-d440b1e076f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:04.973840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadf757-4615-4657-b68e-d440b1e076f0 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:04.985659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 740/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:07.371896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd55c-651b-40c1-b271-19fd2d0517eb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:07.372497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd55c-651b-40c1-b271-19fd2d0517eb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:07.372497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd55c-651b-40c1-b271-19fd2d0517eb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:07.374620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:07.380021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceccda-579a-4e20-bb86-470443f7bb6c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:07.380508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceccda-579a-4e20-bb86-470443f7bb6c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:07.380605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceccda-579a-4e20-bb86-470443f7bb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:07.399438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-01ceccda-579a-4e20-bb86-470443f7bb6c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:07.400689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-01ceccda-579a-4e20-bb86-470443f7bb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:07.408651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:38:07.446165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd55c-651b-40c1-b271-19fd2d0517eb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:07.458770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd55c-651b-40c1-b271-19fd2d0517eb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:07.462091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4d722-f5e9-4cd3-a7dc-5019913b93e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:07.462731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4d722-f5e9-4cd3-a7dc-5019913b93e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:07.463011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4d722-f5e9-4cd3-a7dc-5019913b93e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:07.463287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4d722-f5e9-4cd3-a7dc-5019913b93e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:07.472843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4d722-f5e9-4cd3-a7dc-5019913b93e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:07.474281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4d722-f5e9-4cd3-a7dc-5019913b93e1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:07.483916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 723/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:09.746695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50297f4f-082a-4048-b479-f189bd2a1011 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:09.747276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50297f4f-082a-4048-b479-f189bd2a1011 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:09.747276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50297f4f-082a-4048-b479-f189bd2a1011 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:09.749325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:09.754815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bccff3-7902-4a73-99d3-2c35d03d9268 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:09.755254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bccff3-7902-4a73-99d3-2c35d03d9268 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:09.755378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bccff3-7902-4a73-99d3-2c35d03d9268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:09.773875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-58bccff3-7902-4a73-99d3-2c35d03d9268 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:09.775069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58bccff3-7902-4a73-99d3-2c35d03d9268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:09.783695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:09.821793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50297f4f-082a-4048-b479-f189bd2a1011 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:09.836099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50297f4f-082a-4048-b479-f189bd2a1011 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:09.841174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517d8ee-c43f-4175-8889-a066a526589e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:09.842082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517d8ee-c43f-4175-8889-a066a526589e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:09.842499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517d8ee-c43f-4175-8889-a066a526589e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:09.842912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517d8ee-c43f-4175-8889-a066a526589e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:09.856981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517d8ee-c43f-4175-8889-a066a526589e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:09.858732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517d8ee-c43f-4175-8889-a066a526589e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:09.870701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 743/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:12.106021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc13ea-5a99-4951-afc5-ad2faf460c08 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:12.106601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc13ea-5a99-4951-afc5-ad2faf460c08 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:12.106601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc13ea-5a99-4951-afc5-ad2faf460c08 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:12.108275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:12.113669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5dd46-51bf-4754-bdf5-ac6162167115 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:12.114102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5dd46-51bf-4754-bdf5-ac6162167115 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:12.114234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5dd46-51bf-4754-bdf5-ac6162167115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:12.132374 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b8b5dd46-51bf-4754-bdf5-ac6162167115 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:12.133584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b8b5dd46-51bf-4754-bdf5-ac6162167115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:12.142527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:12.178692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc13ea-5a99-4951-afc5-ad2faf460c08 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:12.180410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28bc13ea-5a99-4951-afc5-ad2faf460c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:12.193701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc13ea-5a99-4951-afc5-ad2faf460c08 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:12.197229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a5a8e-4f00-4f64-97b4-21637fdf0f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:12.197865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a5a8e-4f00-4f64-97b4-21637fdf0f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:12.198149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a5a8e-4f00-4f64-97b4-21637fdf0f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:12.198429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a5a8e-4f00-4f64-97b4-21637fdf0f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:12.208571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a5a8e-4f00-4f64-97b4-21637fdf0f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:12.210044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a5a8e-4f00-4f64-97b4-21637fdf0f69 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:12.219515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 726/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:22.127286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=17 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:38:22.215207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:38:46.057394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ce4-3fd9-4243-8b22-82df0d236ebc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:46.057952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ce4-3fd9-4243-8b22-82df0d236ebc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:46.057952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ce4-3fd9-4243-8b22-82df0d236ebc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:46.059270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 745/1471] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:46.064811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee196e6-3683-4615-a1b5-cde272a8f182 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:46.065285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee196e6-3683-4615-a1b5-cde272a8f182 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:46.065385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee196e6-3683-4615-a1b5-cde272a8f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:46.084398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dee196e6-3683-4615-a1b5-cde272a8f182 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:46.085607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dee196e6-3683-4615-a1b5-cde272a8f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:46.094093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 727/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:46.140989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf2d4-249a-461b-b74d-b3bdf63a836f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:46.141286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf2d4-249a-461b-b74d-b3bdf63a836f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:46.141574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf2d4-249a-461b-b74d-b3bdf63a836f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:46.142956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 746/1473] 10.222.0.16 () {54 vars in 962 bytes} [Thu May 26 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:46.147753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148cf5-ba60-48e4-a087-9eae55e2ce6f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:46.148043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148cf5-ba60-48e4-a087-9eae55e2ce6f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:46.148345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148cf5-ba60-48e4-a087-9eae55e2ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:46.167357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-52148cf5-ba60-48e4-a087-9eae55e2ce6f None None] MFA Rules not processed for user `4e8552f561ce4b4fa837f907a58816aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:46.168801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-52148cf5-ba60-48e4-a087-9eae55e2ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:46.175190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 728/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Thu May 26 13:38:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:46.182336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bf2d4-249a-461b-b74d-b3bdf63a836f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:46.193037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bf2d4-249a-461b-b74d-b3bdf63a836f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:46.196577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a03dd-882f-4c47-8d54-155adfba810d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:46.197228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a03dd-882f-4c47-8d54-155adfba810d service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:46.197511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a03dd-882f-4c47-8d54-155adfba810d service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:46.197794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a03dd-882f-4c47-8d54-155adfba810d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:46.207692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a03dd-882f-4c47-8d54-155adfba810d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:46.209162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a03dd-882f-4c47-8d54-155adfba810d service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:46.219654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 747/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:48.566876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e99b-ce86-4881-8258-98fe063f984d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:48.567486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e99b-ce86-4881-8258-98fe063f984d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:48.567486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e99b-ce86-4881-8258-98fe063f984d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:48.568957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 729/1476] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:48.574871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ebd82-b6ef-4eab-9637-efb1a9ba4db4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:48.575367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ebd82-b6ef-4eab-9637-efb1a9ba4db4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:48.575509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ebd82-b6ef-4eab-9637-efb1a9ba4db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:48.594704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-712ebd82-b6ef-4eab-9637-efb1a9ba4db4 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:48.595918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-712ebd82-b6ef-4eab-9637-efb1a9ba4db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:48.603966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 748/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:48.627221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e99b-ce86-4881-8258-98fe063f984d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:48.640477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e99b-ce86-4881-8258-98fe063f984d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:48.644830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:48.645471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:48.645750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:48.646023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:48.646739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:48.648552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:38:48.648552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:38:48.648552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:38:48.649689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283b3f2-d7fb-49be-ac39-5e943e93474d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:48.668682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 730/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu May 26 13:38:48 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:38:50.953213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c513-1fe3-45bc-93e9-9d7f5942f0b6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:50.953753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c513-1fe3-45bc-93e9-9d7f5942f0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:50.953874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c513-1fe3-45bc-93e9-9d7f5942f0b6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:50.955187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 749/1479] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:50.960683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09d6c-a9c1-4351-82e1-5dc37321ad7c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:50.961124 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09d6c-a9c1-4351-82e1-5dc37321ad7c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:50.961250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09d6c-a9c1-4351-82e1-5dc37321ad7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:50.979641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c4f09d6c-a9c1-4351-82e1-5dc37321ad7c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:50.981560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4f09d6c-a9c1-4351-82e1-5dc37321ad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:50.989445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 731/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:51.033382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c513-1fe3-45bc-93e9-9d7f5942f0b6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:51.049426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c513-1fe3-45bc-93e9-9d7f5942f0b6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:51.054913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795f472-831e-4cbe-be4c-735408befb7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:51.055583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f472-831e-4cbe-be4c-735408befb7b service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:51.055870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f472-831e-4cbe-be4c-735408befb7b service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:51.056151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795f472-831e-4cbe-be4c-735408befb7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:51.064875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f472-831e-4cbe-be4c-735408befb7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:51.066467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795f472-831e-4cbe-be4c-735408befb7b service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:51.076002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 750/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:53.489237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e3998-a94c-498f-8a4b-d770dfaa36f0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:53.489772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e3998-a94c-498f-8a4b-d770dfaa36f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:53.489772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e3998-a94c-498f-8a4b-d770dfaa36f0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:53.491110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 732/1482] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:53.497233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb0672-aab1-43a1-8930-5f9702fd7de5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:53.497683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb0672-aab1-43a1-8930-5f9702fd7de5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:53.497807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb0672-aab1-43a1-8930-5f9702fd7de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:53.515535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-28fb0672-aab1-43a1-8930-5f9702fd7de5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:53.516778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28fb0672-aab1-43a1-8930-5f9702fd7de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:53.524577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 751/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:38:53.561286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e3998-a94c-498f-8a4b-d770dfaa36f0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:53.571888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e3998-a94c-498f-8a4b-d770dfaa36f0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:53.575825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b56e1-34d4-4371-977a-3625fcae14bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:53.576611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b56e1-34d4-4371-977a-3625fcae14bf service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:53.576954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b56e1-34d4-4371-977a-3625fcae14bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:53.577301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b56e1-34d4-4371-977a-3625fcae14bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:53.587907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b56e1-34d4-4371-977a-3625fcae14bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:53.589482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b56e1-34d4-4371-977a-3625fcae14bf service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:53.599004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 733/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:55.896414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af91eda-83be-4891-afef-d0c12e1f0389 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:55.896988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af91eda-83be-4891-afef-d0c12e1f0389 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:55.896988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af91eda-83be-4891-afef-d0c12e1f0389 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:55.898364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 752/1485] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:55.903981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4cd3b-b462-4696-a1ae-eb226dbd8409 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:55.904475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4cd3b-b462-4696-a1ae-eb226dbd8409 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:55.904583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4cd3b-b462-4696-a1ae-eb226dbd8409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:55.924342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-54b4cd3b-b462-4696-a1ae-eb226dbd8409 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:55.925550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54b4cd3b-b462-4696-a1ae-eb226dbd8409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:55.934280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 734/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:55.971694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af91eda-83be-4891-afef-d0c12e1f0389 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:55.983344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af91eda-83be-4891-afef-d0c12e1f0389 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:55.986785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19143ce9-a96b-4d68-b28b-10d0e260c37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:55.987419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19143ce9-a96b-4d68-b28b-10d0e260c37e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:55.987706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19143ce9-a96b-4d68-b28b-10d0e260c37e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:55.987976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19143ce9-a96b-4d68-b28b-10d0e260c37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:55.997901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19143ce9-a96b-4d68-b28b-10d0e260c37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:55.999336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19143ce9-a96b-4d68-b28b-10d0e260c37e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:56.009056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 753/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:38:58.401214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8efb0-395d-48d6-bf87-3cda8fe90eb7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:58.402684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8efb0-395d-48d6-bf87-3cda8fe90eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:58.402684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8efb0-395d-48d6-bf87-3cda8fe90eb7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:58.403897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 735/1488] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:38:58.409523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdefdd-d7cf-47fc-a55a-7348c9432471 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:58.410014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdefdd-d7cf-47fc-a55a-7348c9432471 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:58.410118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdefdd-d7cf-47fc-a55a-7348c9432471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:58.428322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4bcdefdd-d7cf-47fc-a55a-7348c9432471 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:38:58.429586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4bcdefdd-d7cf-47fc-a55a-7348c9432471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:38:58.439450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 754/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:38:58.486544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8efb0-395d-48d6-bf87-3cda8fe90eb7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:38:58.497467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8efb0-395d-48d6-bf87-3cda8fe90eb7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:38:58.501034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7b377-fe95-4e1a-9056-43721c1d5933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:38:58.501673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7b377-fe95-4e1a-9056-43721c1d5933 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:38:58.501959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7b377-fe95-4e1a-9056-43721c1d5933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:38:58.502241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7b377-fe95-4e1a-9056-43721c1d5933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:38:58.511803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7b377-fe95-4e1a-9056-43721c1d5933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:38:58.513276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7b377-fe95-4e1a-9056-43721c1d5933 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:38:58.522674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 736/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:39:00.973780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d2071-73a7-4630-b071-bda231a0c635 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:39:00.974331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d2071-73a7-4630-b071-bda231a0c635 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:39:00.974331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d2071-73a7-4630-b071-bda231a0c635 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:39:00.975727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 755/1491] 10.222.0.16 () {54 vars in 926 bytes} [Thu May 26 13:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:39:00.981216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7d44-7756-4483-8789-dcf082bbd2a3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:39:00.981695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7d44-7756-4483-8789-dcf082bbd2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:39:00.981804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7d44-7756-4483-8789-dcf082bbd2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:39:01.000455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-beaa7d44-7756-4483-8789-dcf082bbd2a3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:39:01.001698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-beaa7d44-7756-4483-8789-dcf082bbd2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:39:01.009764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 737/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Thu May 26 13:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:39:01.049600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d2071-73a7-4630-b071-bda231a0c635 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:39:01.061046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d2071-73a7-4630-b071-bda231a0c635 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:39:01.064702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c44cf8-8762-4ef6-ae9e-11e2607fee14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:39:01.065356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c44cf8-8762-4ef6-ae9e-11e2607fee14 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:39:01.065641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c44cf8-8762-4ef6-ae9e-11e2607fee14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:39:01.065922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c44cf8-8762-4ef6-ae9e-11e2607fee14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:39:01.076125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c44cf8-8762-4ef6-ae9e-11e2607fee14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:39:01.077872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c44cf8-8762-4ef6-ae9e-11e2607fee14 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:39:01.087977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 756/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:39:10.995538 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:39:11.083342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=33 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:40:50.420173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.420818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.420818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.440633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:40:50.441859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:50.450938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 738/1494] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:40:50.461087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.461630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.461738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.479727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:40:50.480948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:50.488700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 757/1495] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:40:50.493654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:50.503312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79060aa9-2293-4c2c-a728-0e8259fb723d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:50.508645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:50.509300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.509933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.510213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.510938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:40:50.511249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:50.512283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:50.512283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:40:50.512283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:40:50.513188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:50.526201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 739/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:40:50.534348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:50.543823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b7e17-e2be-4f8b-bc86-b2ad81cbeabd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:50.548743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:50.549492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.549779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.550055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.550778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:50.553010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:50.553010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:40:50.553010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:40:50.554270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:50.595145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 758/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:40:50 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:40:50.600083 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:50.610703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa4d1f-2249-4c25-8606-7838cc7e0734 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:50.666464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:50.667144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.667431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.667720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.668451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:50.669485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:50.669485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:40:50.669485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:40:50.670700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:50.691554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 740/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:40:50 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:40:50.696558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:50.710487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6524352-5ae8-462e-8418-5582fe6c1e09 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:50.715125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:50.715795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.716076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.716379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.717110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:50.718127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:50.718127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:40:50.718127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:40:50.719007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:50.732912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 759/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:40:50.737062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:50.748817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68864d18-3e6a-4a12-805b-581c808ca329 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:50.753601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:50.754271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.754555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.754832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.755612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:50.756676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:50.756676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:40:50.756676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:40:50.757570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78ea9-8b7f-47c0-b6cd-44cfe7dc3db9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:50.772244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 741/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:40:50.777045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:50.788736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f913e4-3d41-45b2-8b24-a0ad1aecf16d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:50.793525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:50.794462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.794752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.795033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] PATH_INFO: `/v3/projects/58c77078a8d940348a2d6f00d9a61c8d/users/d6674d016d1947a4abb7dd565193165e/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.811727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c77078a8d940348a2d6f00d9a61c8d, user_id=d6674d016d1947a4abb7dd565193165e, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:50.812800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:50.812800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:40:50.812800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:40:50.814758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62216837-203a-4e46-a24e-aa24c21555be admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:50.829309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 760/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:40:50 2022] PUT /identity/v3/projects/58c77078a8d940348a2d6f00d9a61c8d/users/d6674d016d1947a4abb7dd565193165e/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:40:50.841891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b756e-714e-44e0-8741-95c5b5b19a2d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.842363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b756e-714e-44e0-8741-95c5b5b19a2d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.842742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b756e-714e-44e0-8741-95c5b5b19a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.875462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9c3b756e-714e-44e0-8741-95c5b5b19a2d None None] MFA Rules not processed for user `d6674d016d1947a4abb7dd565193165e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:40:50.876749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c3b756e-714e-44e0-8741-95c5b5b19a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:50.946041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 742/1502] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:40:50.961502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4746-5d31-4cee-98a0-b09ed5470041 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.961828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4746-5d31-4cee-98a0-b09ed5470041 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.962109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4746-5d31-4cee-98a0-b09ed5470041 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.963479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 761/1503] 10.222.0.16 () {54 vars in 963 bytes} [Thu May 26 13:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:40:50.968726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90e3d6-f8ba-43db-808d-d52d900d7c0c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:50.969022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90e3d6-f8ba-43db-808d-d52d900d7c0c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:50.969313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90e3d6-f8ba-43db-808d-d52d900d7c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:50.989164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef90e3d6-f8ba-43db-808d-d52d900d7c0c None None] MFA Rules not processed for user `867e9b57fa7947e3afcbd974ed887969`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:40:50.990654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef90e3d6-f8ba-43db-808d-d52d900d7c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:51.037804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:40:51.046568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 743/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Thu May 26 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:40:51.054618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4746-5d31-4cee-98a0-b09ed5470041 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:51.065407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4746-5d31-4cee-98a0-b09ed5470041 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:51.069178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:51.069830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:51.070117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:51.070400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:51.079908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:51.081565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:51.092712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 762/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:40:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:40:51.136617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c84f-c2d0-4e59-affa-b62a8663ace8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:51.136904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c84f-c2d0-4e59-affa-b62a8663ace8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:51.137180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c84f-c2d0-4e59-affa-b62a8663ace8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:51.138518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 744/1506] 10.222.0.16 () {52 vars in 829 bytes} [Thu May 26 13:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:40:51.142826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:51.209356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e702e-97bc-46d3-9d23-c824938e26da service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:51.214108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:51.215001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:51.215284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:51.215584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] PATH_INFO: `/v3/users/d6674d016d1947a4abb7dd565193165e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:51.216819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6674d016d1947a4abb7dd565193165e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:51.217870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:51.217870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:40:51.217870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:40:51.219111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:51.291553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6674d016d1947a4abb7dd565193165e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:51.292268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6674d016d1947a4abb7dd565193165e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:51.307630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6674d016d1947a4abb7dd565193165e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:51.308627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6674d016d1947a4abb7dd565193165e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:51.321103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6674d016d1947a4abb7dd565193165e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:51.344730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:40:51.354800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 763/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:40:51 2022] DELETE /identity/v3/users/d6674d016d1947a4abb7dd565193165e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:40:51.359434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:51.359730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:51.360018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:51.379523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:40:51.381490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:51.447093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 745/1508] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:40:51.457858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:51.468401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:51.534298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4653-15d5-4b25-8d04-cfb8ef4e8e4a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:51.539010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baa466-46da-44b3-aac2-afe5d85730de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:51.539677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baa466-46da-44b3-aac2-afe5d85730de service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:51.539961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baa466-46da-44b3-aac2-afe5d85730de service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:51.540234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baa466-46da-44b3-aac2-afe5d85730de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:51.549893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baa466-46da-44b3-aac2-afe5d85730de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:51.551368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baa466-46da-44b3-aac2-afe5d85730de service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:51.563708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 764/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:40:51.982499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:40:51.984078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:40:52.001104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04eaf-f38d-4dfe-9a50-6353fd1d5a25 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:40:52.011079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:40:52.011079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:40:52.011079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:40:52.011079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] PATH_INFO: `/v3/projects/58c77078a8d940348a2d6f00d9a61c8d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:40:52.011697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c77078a8d940348a2d6f00d9a61c8d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:40:52.011697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:40:52.011697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:40:52.011697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:40:52.012798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:40:52.097658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c77078a8d940348a2d6f00d9a61c8d', 'request_id': 'req-5233026f-21bd-4748-a4ad-32c46333d259'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:52.104778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c77078a8d940348a2d6f00d9a61c8d', 'request_id': 'req-5233026f-21bd-4748-a4ad-32c46333d259'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:52.115359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c77078a8d940348a2d6f00d9a61c8d', 'request_id': 'req-5233026f-21bd-4748-a4ad-32c46333d259'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:52.137368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] The token cache is being invalidate because project 58c77078a8d940348a2d6f00d9a61c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:40:52.137664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:52.138357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5233026f-21bd-4748-a4ad-32c46333d259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:40:52.140044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 746/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:40:51 2022] DELETE /identity/v3/projects/58c77078a8d940348a2d6f00d9a61c8d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:41:01.558837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:41:02.133739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:34.938701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157bf85-6134-497f-a09d-18bed0effcc8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.939262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157bf85-6134-497f-a09d-18bed0effcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.939262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157bf85-6134-497f-a09d-18bed0effcc8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.940778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:34.945686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f1c2-ef5b-4685-b587-5630d6b6065d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:34.946111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f1c2-ef5b-4685-b587-5630d6b6065d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:34.946248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f1c2-ef5b-4685-b587-5630d6b6065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:34.966176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d363f1c2-ef5b-4685-b587-5630d6b6065d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:34.967386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d363f1c2-ef5b-4685-b587-5630d6b6065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:35.024379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:35.196530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157bf85-6134-497f-a09d-18bed0effcc8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:35.198217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8157bf85-6134-497f-a09d-18bed0effcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:35.275712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157bf85-6134-497f-a09d-18bed0effcc8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:35.282051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1998522-0619-44a0-a337-330f35614c55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:35.282883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1998522-0619-44a0-a337-330f35614c55 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:35.283171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1998522-0619-44a0-a337-330f35614c55 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:35.283455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1998522-0619-44a0-a337-330f35614c55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:35.292775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1998522-0619-44a0-a337-330f35614c55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:35.294414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1998522-0619-44a0-a337-330f35614c55 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:35.305094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 766/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:42:36.201739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70477035-7b56-4575-8f15-42de33639173 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.202353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70477035-7b56-4575-8f15-42de33639173 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.202353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70477035-7b56-4575-8f15-42de33639173 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.204417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:36.209388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5c72-22bd-4aa3-86e4-327c8f6ae839 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.209837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5c72-22bd-4aa3-86e4-327c8f6ae839 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.209937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e5c72-22bd-4aa3-86e4-327c8f6ae839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.228142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-959e5c72-22bd-4aa3-86e4-327c8f6ae839 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:36.229376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-959e5c72-22bd-4aa3-86e4-327c8f6ae839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:36.237444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:36.392085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70477035-7b56-4575-8f15-42de33639173 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:36.406553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70477035-7b56-4575-8f15-42de33639173 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:36.412323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af77e-b8bb-485c-9937-03fddca24fa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:36.412969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af77e-b8bb-485c-9937-03fddca24fa3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.413269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af77e-b8bb-485c-9937-03fddca24fa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.413560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af77e-b8bb-485c-9937-03fddca24fa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.422052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3af77e-b8bb-485c-9937-03fddca24fa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:36.423699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3af77e-b8bb-485c-9937-03fddca24fa3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:36.433766 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 749/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:42:36.561007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ab02-8795-4c3c-b31f-5ae468191917 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.561305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ab02-8795-4c3c-b31f-5ae468191917 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.561593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ab02-8795-4c3c-b31f-5ae468191917 None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.563772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 768/1517] 10.222.0.57 () {54 vars in 925 bytes} [Thu May 26 13:42:36 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:42:36.570277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab7f43-e229-46b8-aacc-ea20d150e473 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:36.571212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab7f43-e229-46b8-aacc-ea20d150e473 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:36.571510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab7f43-e229-46b8-aacc-ea20d150e473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:36.591176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-09ab7f43-e229-46b8-aacc-ea20d150e473 None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:36.592408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-09ab7f43-e229-46b8-aacc-ea20d150e473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:36.659179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 750/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 26 13:42:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:40.707554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b9b6d-7529-41e9-a538-9bd85c0dc060 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.708507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b9b6d-7529-41e9-a538-9bd85c0dc060 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.708507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b9b6d-7529-41e9-a538-9bd85c0dc060 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.710560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 769/1519] 10.222.0.16 () {54 vars in 960 bytes} [Thu May 26 13:42:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:40.718166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b1957-e82a-459a-a595-10545a715c0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.719883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b1957-e82a-459a-a595-10545a715c0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.719883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b1957-e82a-459a-a595-10545a715c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.740491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e54b1957-e82a-459a-a595-10545a715c0f None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:40.741956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e54b1957-e82a-459a-a595-10545a715c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:40.752072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 751/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Thu May 26 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:40.762167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b9b6d-7529-41e9-a538-9bd85c0dc060 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:40.776435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b9b6d-7529-41e9-a538-9bd85c0dc060 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:40.782084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f05c70-7ddc-4be7-b00a-1e16235301c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:40.782829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f05c70-7ddc-4be7-b00a-1e16235301c1 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:40.783116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f05c70-7ddc-4be7-b00a-1e16235301c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:40.783397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f05c70-7ddc-4be7-b00a-1e16235301c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:40.794111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f05c70-7ddc-4be7-b00a-1e16235301c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:40.796021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f05c70-7ddc-4be7-b00a-1e16235301c1 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:40.807287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 770/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:42:47.749765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f061-eaa8-4e81-9917-aca412f7e6ce None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.750359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f061-eaa8-4e81-9917-aca412f7e6ce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.750359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f061-eaa8-4e81-9917-aca412f7e6ce None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.751769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:47.756865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc84e70-2826-4600-bcdb-15b9344a4f90 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.757315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc84e70-2826-4600-bcdb-15b9344a4f90 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.757419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc84e70-2826-4600-bcdb-15b9344a4f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.775598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3dc84e70-2826-4600-bcdb-15b9344a4f90 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:47.777056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3dc84e70-2826-4600-bcdb-15b9344a4f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:47.785936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:47.942172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924f061-eaa8-4e81-9917-aca412f7e6ce None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:47.961752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924f061-eaa8-4e81-9917-aca412f7e6ce None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:47.967050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439e7b-fce8-400a-a491-677f4a4ff596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:47.967705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439e7b-fce8-400a-a491-677f4a4ff596 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:47.967982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439e7b-fce8-400a-a491-677f4a4ff596 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:47.968998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439e7b-fce8-400a-a491-677f4a4ff596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:47.977774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439e7b-fce8-400a-a491-677f4a4ff596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:47.979474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439e7b-fce8-400a-a491-677f4a4ff596 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:47.989258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 753/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:42:48.863818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb3f2d-dd41-4074-834c-6b1b6b7343c8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.864383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb3f2d-dd41-4074-834c-6b1b6b7343c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.864485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb3f2d-dd41-4074-834c-6b1b6b7343c8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.865805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 13:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:48.871091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db6828-307d-4d8f-b7ae-77c4b6e36c6b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:48.871548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db6828-307d-4d8f-b7ae-77c4b6e36c6b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:48.871746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db6828-307d-4d8f-b7ae-77c4b6e36c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:48.891725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e5db6828-307d-4d8f-b7ae-77c4b6e36c6b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:48.892981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e5db6828-307d-4d8f-b7ae-77c4b6e36c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:48.901184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 13:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:42:49.054975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb3f2d-dd41-4074-834c-6b1b6b7343c8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:49.069793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb3f2d-dd41-4074-834c-6b1b6b7343c8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:49.075830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ddafe-b578-44ef-a6a7-56d7c218b3b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:49.076771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ddafe-b578-44ef-a6a7-56d7c218b3b9 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:49.077061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ddafe-b578-44ef-a6a7-56d7c218b3b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:49.077347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ddafe-b578-44ef-a6a7-56d7c218b3b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:49.085782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ddafe-b578-44ef-a6a7-56d7c218b3b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:49.087448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ddafe-b578-44ef-a6a7-56d7c218b3b9 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:49.098680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 773/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:42:53.257016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7045ded-6ccc-4cf1-88a9-18a04c7c4665 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:53.257591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7045ded-6ccc-4cf1-88a9-18a04c7c4665 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:53.257591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7045ded-6ccc-4cf1-88a9-18a04c7c4665 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:53.258998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:53.264555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36693d-d036-407f-b442-92b377d2265a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:53.265532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36693d-d036-407f-b442-92b377d2265a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:53.265773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36693d-d036-407f-b442-92b377d2265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:53.284706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0a36693d-d036-407f-b442-92b377d2265a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:53.285937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0a36693d-d036-407f-b442-92b377d2265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:53.293981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:53.332669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7045ded-6ccc-4cf1-88a9-18a04c7c4665 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:53.334528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7045ded-6ccc-4cf1-88a9-18a04c7c4665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:53.393759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7045ded-6ccc-4cf1-88a9-18a04c7c4665 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:53.397257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449bb81-0776-4fc2-a560-10d4c3cdb43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:53.397893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449bb81-0776-4fc2-a560-10d4c3cdb43a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:53.398174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449bb81-0776-4fc2-a560-10d4c3cdb43a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:53.398450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449bb81-0776-4fc2-a560-10d4c3cdb43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:53.407559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449bb81-0776-4fc2-a560-10d4c3cdb43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:53.409128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449bb81-0776-4fc2-a560-10d4c3cdb43a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:53.414265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:42:53.420753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 756/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:42:55.704058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40081383-a62c-4b54-99d5-4d1741de434d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.704632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40081383-a62c-4b54-99d5-4d1741de434d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.704749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40081383-a62c-4b54-99d5-4d1741de434d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.706318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:55.712065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aeaf0-3e23-482d-9625-8564e71b99df None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.712575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aeaf0-3e23-482d-9625-8564e71b99df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.712682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aeaf0-3e23-482d-9625-8564e71b99df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.730994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b20aeaf0-3e23-482d-9625-8564e71b99df None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:55.732196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b20aeaf0-3e23-482d-9625-8564e71b99df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:55.740493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:42:55.778789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40081383-a62c-4b54-99d5-4d1741de434d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:42:55.790189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40081383-a62c-4b54-99d5-4d1741de434d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:42:55.794505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aef44-f966-4bd9-b238-93d94e44e82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:42:55.795155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aef44-f966-4bd9-b238-93d94e44e82f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:55.795437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aef44-f966-4bd9-b238-93d94e44e82f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:55.795718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aef44-f966-4bd9-b238-93d94e44e82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:55.805730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aef44-f966-4bd9-b238-93d94e44e82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:42:55.807232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aef44-f966-4bd9-b238-93d94e44e82f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:42:55.818035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 776/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:42:59.712255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f75b39-601a-4cc0-a71b-0fadd1080cc2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.712854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f75b39-601a-4cc0-a71b-0fadd1080cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.712854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f75b39-601a-4cc0-a71b-0fadd1080cc2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.714256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:42:59.719963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27538de1-dcf3-409c-bb2d-d02d2926d031 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:42:59.720407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27538de1-dcf3-409c-bb2d-d02d2926d031 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:42:59.720540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27538de1-dcf3-409c-bb2d-d02d2926d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:42:59.738925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-27538de1-dcf3-409c-bb2d-d02d2926d031 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:42:59.740114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27538de1-dcf3-409c-bb2d-d02d2926d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:42:59.747975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:43:00.037548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f75b39-601a-4cc0-a71b-0fadd1080cc2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:00.039298 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e9f75b39-601a-4cc0-a71b-0fadd1080cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:00.101552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f75b39-601a-4cc0-a71b-0fadd1080cc2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:00.105173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356b691-bb6b-4d34-b1e7-792619a17c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:00.105875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b691-bb6b-4d34-b1e7-792619a17c49 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:00.106165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b691-bb6b-4d34-b1e7-792619a17c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:00.106450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b691-bb6b-4d34-b1e7-792619a17c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:00.117246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356b691-bb6b-4d34-b1e7-792619a17c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:00.118710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356b691-bb6b-4d34-b1e7-792619a17c49 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:00.129373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 759/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:43:02.461151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b2458-c341-4b16-b94e-3998de01cbca None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.462150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b2458-c341-4b16-b94e-3998de01cbca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.462150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b2458-c341-4b16-b94e-3998de01cbca None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.463888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 13:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:43:02.470975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8e778-f67c-4fd0-8b3e-53988242510c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.471568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8e778-f67c-4fd0-8b3e-53988242510c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.471727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8e778-f67c-4fd0-8b3e-53988242510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.493654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4fb8e778-f67c-4fd0-8b3e-53988242510c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:43:02.494889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4fb8e778-f67c-4fd0-8b3e-53988242510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:02.503008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 13:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:43:02.786756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b2458-c341-4b16-b94e-3998de01cbca None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:43:02.788531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7b9b2458-c341-4b16-b94e-3998de01cbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:43:02.796936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=65 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:02.803649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b2458-c341-4b16-b94e-3998de01cbca None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:43:02.807284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedb1a4-8f5c-43a5-9eff-47af02e1cb65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:43:02.807947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedb1a4-8f5c-43a5-9eff-47af02e1cb65 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:43:02.808234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedb1a4-8f5c-43a5-9eff-47af02e1cb65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:43:02.808554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedb1a4-8f5c-43a5-9eff-47af02e1cb65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:43:02.818532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedb1a4-8f5c-43a5-9eff-47af02e1cb65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:43:02.819999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedb1a4-8f5c-43a5-9eff-47af02e1cb65 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:43:02.830912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 779/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:43:12.487813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:43:12.825906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:34.394551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.395193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.395193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.441373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:34.443422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:34.453508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 761/1540] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:34.463976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.464574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.464665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.500105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:34.506266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:34.518691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 780/1541] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:34.525138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:34.540564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0f268-e012-4e26-95bc-9e037f5a794a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:34.554342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:34.555470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.556417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.557472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.558463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:34.559001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:34.574886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:34.595084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 762/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:34.603532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:34.620005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ff4f3-07ae-47a0-8a41-8f8cdab7fcda None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:34.629039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:34.629743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.630034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.630310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.631032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:34.633062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:34.680657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 781/1543] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:34 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:34.686324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:34.702936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de7221-c203-45ec-bf4e-217a57e0fe50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:34.778546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:34.779663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.780181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.780726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.781849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:34.784213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:34.817924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 763/1544] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:34 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:34.821418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:34.839267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1029aa-fb96-485d-80fd-b6151511568d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:34.845509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922696b-25b9-41c6-9628-d7295523bf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:34.846631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922696b-25b9-41c6-9628-d7295523bf73 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.847155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922696b-25b9-41c6-9628-d7295523bf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.847676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922696b-25b9-41c6-9628-d7295523bf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.853824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922696b-25b9-41c6-9628-d7295523bf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:34.856107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922696b-25b9-41c6-9628-d7295523bf73 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:34.870386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 782/1545] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:34.874661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:34.887415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9270389-2cae-493d-aba0-b9c543f5c1de admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:34.897936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.898273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.898514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.914639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b499-cd34-4a10-879a-8f2f008c3e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:34.915345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b499-cd34-4a10-879a-8f2f008c3e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.915634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b499-cd34-4a10-879a-8f2f008c3e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.915948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b499-cd34-4a10-879a-8f2f008c3e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.919536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b499-cd34-4a10-879a-8f2f008c3e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:34.930590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b499-cd34-4a10-879a-8f2f008c3e2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:34.949288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:34.952984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:34.962212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 764/1546] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:44:34.963779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04db211-f6dc-48df-a198-9fa3b3d90848 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.963941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 783/1547] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:34.964039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04db211-f6dc-48df-a198-9fa3b3d90848 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.964292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04db211-f6dc-48df-a198-9fa3b3d90848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:34.966129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:34.979360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557fe53-f26a-4606-8129-b3884fddfae5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:34.987614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bd822-6f78-41f2-a561-020b820403c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:34.988571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bd822-6f78-41f2-a561-020b820403c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:34.988853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bd822-6f78-41f2-a561-020b820403c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:34.989133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bd822-6f78-41f2-a561-020b820403c8 admin admin] PATH_INFO: `/v3/projects/1ce476b2a1764f56b9780f4a5c75ac3d/users/e13f5cad16824554a1fb32ed9a915ae9/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.004052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e04db211-f6dc-48df-a198-9fa3b3d90848 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.005330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e04db211-f6dc-48df-a198-9fa3b3d90848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.011831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bd822-6f78-41f2-a561-020b820403c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce476b2a1764f56b9780f4a5c75ac3d, user_id=e13f5cad16824554a1fb32ed9a915ae9, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.014820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bd822-6f78-41f2-a561-020b820403c8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.025076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 765/1548] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.030521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47bd0-13db-4129-a5a6-1dede88dbb50 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.031439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47bd0-13db-4129-a5a6-1dede88dbb50 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.031729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47bd0-13db-4129-a5a6-1dede88dbb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.034925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 784/1549] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:34 2022] PUT /identity/v3/projects/1ce476b2a1764f56b9780f4a5c75ac3d/users/e13f5cad16824554a1fb32ed9a915ae9/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:44:35.037929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783290b-4c2c-496b-a354-5afd4a608d4e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.037929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783290b-4c2c-496b-a354-5afd4a608d4e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.038164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783290b-4c2c-496b-a354-5afd4a608d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.066645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-abc47bd0-13db-4129-a5a6-1dede88dbb50 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.067885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abc47bd0-13db-4129-a5a6-1dede88dbb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.070225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3783290b-4c2c-496b-a354-5afd4a608d4e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.071424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3783290b-4c2c-496b-a354-5afd4a608d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.135173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 766/1550] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.136530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e82cf6-abcf-4ac4-b326-6ae4b4f9321b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.136828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e82cf6-abcf-4ac4-b326-6ae4b4f9321b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.137112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e82cf6-abcf-4ac4-b326-6ae4b4f9321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.145551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 785/1551] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.147289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77202-1bbc-4012-9579-f741f67fb07f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.147502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77202-1bbc-4012-9579-f741f67fb07f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.147771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77202-1bbc-4012-9579-f741f67fb07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.156135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c7e82cf6-abcf-4ac4-b326-6ae4b4f9321b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.157402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7e82cf6-abcf-4ac4-b326-6ae4b4f9321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.165505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 767/1552] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.167501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743c1a9-960d-4d1d-8e0b-d12a31fd7706 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.167797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743c1a9-960d-4d1d-8e0b-d12a31fd7706 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.168064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743c1a9-960d-4d1d-8e0b-d12a31fd7706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.173495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ddc77202-1bbc-4012-9579-f741f67fb07f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.175001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ddc77202-1bbc-4012-9579-f741f67fb07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.184758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 786/1553] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.186215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b8d68-aec1-44f0-9d56-908f585f72ec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.186510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b8d68-aec1-44f0-9d56-908f585f72ec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.186813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b8d68-aec1-44f0-9d56-908f585f72ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.193346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1743c1a9-960d-4d1d-8e0b-d12a31fd7706 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.195641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1743c1a9-960d-4d1d-8e0b-d12a31fd7706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.207220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a99b8d68-aec1-44f0-9d56-908f585f72ec None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.208382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a99b8d68-aec1-44f0-9d56-908f585f72ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.208513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 768/1554] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.209906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.210203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.210494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.220369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 787/1555] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.221795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642fc1a-f383-463b-8232-eefbf405e29d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.222098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642fc1a-f383-463b-8232-eefbf405e29d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.222371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642fc1a-f383-463b-8232-eefbf405e29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.230972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.232190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.240830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 769/1556] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.241730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.242670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2642fc1a-f383-463b-8232-eefbf405e29d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.243854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2642fc1a-f383-463b-8232-eefbf405e29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.252247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c985db-e4a4-4b83-a75a-0b683d8a733b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.253221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 788/1557] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.254414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.254757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.254978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.257173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.257851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.258137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.258411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.259196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.259509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.261374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba26d9-131f-4595-af89-dfbe7b5d60b3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.273160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.274004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 770/1558] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.281313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.281420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.281420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.281420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.282712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 789/1559] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.283509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.294472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350099bf-8514-4cb1-a132-b706aa9acaaa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.298425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.299667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.300599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.301523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.301909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.302356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.303317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.303631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.304859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 13:44:35.304859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:44:35.304859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:44:35.305923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095eaa-10c2-450d-817b-3580626331cc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.309027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 771/1560] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.309639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.325604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:35.328974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 790/1561] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.330306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e5f3-8a12-4ca3-b976-1416a94a7bcf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.330595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e5f3-8a12-4ca3-b976-1416a94a7bcf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.330880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e5f3-8a12-4ca3-b976-1416a94a7bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.348806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-68e0e5f3-8a12-4ca3-b976-1416a94a7bcf None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.350002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68e0e5f3-8a12-4ca3-b976-1416a94a7bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.358711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 791/1562] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.360055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b34fe-50b8-4f53-bfc9-6987cf1dad02 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.360379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b34fe-50b8-4f53-bfc9-6987cf1dad02 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.360658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b34fe-50b8-4f53-bfc9-6987cf1dad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.379994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08bc80-2465-4d79-9ea1-f259a3ac25a7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.380293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ea5b34fe-50b8-4f53-bfc9-6987cf1dad02 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.381499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea5b34fe-50b8-4f53-bfc9-6987cf1dad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.383641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.384290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.384601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.385028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.389825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 792/1563] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.391065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.391353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.391632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.398048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.400704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.410055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:35.411265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:35.411575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 772/1564] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:35.413152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.423810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 793/1565] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:35.423810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.428555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493a831-6706-48a1-92ee-f0f3b16f1663 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.434199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ae37e-8f98-43e1-b60d-64efb31ae3e3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.434489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.435162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.435447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.435727 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.436468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.437876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:44:35.437876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:44:35.437876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:44:35.439432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.440871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.441544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.441969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.442092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.442959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.443432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.445497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.460624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 794/1566] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.461361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.472739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e087-c9f4-4d33-9f6f-1b7fb717ba4a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.480084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 773/1567] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.480084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.482301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.483223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.483718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.484067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.484912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.485302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.486969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.503256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 795/1568] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.504176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.513730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:35.520085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d016b-b2ea-4a18-a16f-1ee4a6f664a1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.571162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfbead-e5f8-4a54-8a0c-102a81de86ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.577871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.578552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.578855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.579113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.579808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.581790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.615642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 774/1569] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.616795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.625458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.626666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.627173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.628099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.629439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.630158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.632595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.649429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 796/1570] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.649429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.660245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cdfa9-7112-4a22-bae9-7dc5aa1c0a45 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.701403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f293c0-7723-4663-8edd-e70829775c72 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.706991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.707713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.707995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.708277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.709367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.709717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.711521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.730875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 775/1571] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.732293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.733146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.733440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.733720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.734453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:35.734909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.736680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.740087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.753544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b919eb-f388-46d4-86d0-c8c783cb0809 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.755605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 797/1572] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:35.756694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.762319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.763122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.763412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.763688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.764407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.773063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653b0b-4e4c-4e6b-96da-6733c10141f0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.775034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.778089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.778773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.779062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.779347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.780053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.782546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:44:35.782546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:44:35.782546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:44:35.783441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.790485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:35.818147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 798/1573] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/users => generated 290 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.818147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.819547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 776/1574] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.820794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.829198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9bcb0-9de6-4d91-8624-32417e278538 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.899637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314294-c254-43a3-81d5-716fab0d1197 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.905729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.906498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.906911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.907069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.907802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.910474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.923303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.924019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.924280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.924606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.925307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.927466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.934137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 777/1575] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.937889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.948832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56500ab7-0b0e-4d29-a8d7-26d7f17383d7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:35.952018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:35.954798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:35.955476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:35.955755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:35.956031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:35.956877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:35.958752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:35.962051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 799/1576] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.963384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:35.991022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 778/1577] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:35.991833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.044235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b3f3c-11d8-40ba-aaf8-d9181154f1b0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.062536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060fa7f-580e-4b23-827a-e2860be5fe3d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.068871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.069613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.069971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.070320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.071108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.074119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.102782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 779/1578] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.103842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.118932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.119750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.120371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.120778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.121574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.123739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.158262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 800/1579] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:35 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.158262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.163479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:36.186220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23934148-dfe9-483c-af82-f0b0f40b2fa3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.229277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168dee9-2cb8-4ea8-9ebb-9b2092f97447 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.237789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.238582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.238883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.239145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.242537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.246942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.271416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.272095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.272416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.272746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.273628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 801/1580] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.274311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.276604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.278642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.286403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47dc2-e30c-4273-8ced-19be83315a1e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.290616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 780/1581] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.291484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.291945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.292726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.293070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.293410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.294250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.295986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.301564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4a5c-7617-4b74-8d4e-0c4e4a04e452 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.309818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.310558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.310848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.311129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.311821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.313034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 802/1582] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.313849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.314109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.330891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62795d2b-d8c9-4a84-957f-8b1db8fc0d73 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.338259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.339034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.339973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.340282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.341081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.342889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 781/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/users => generated 283 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.343989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.344971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.356350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635df40-26b7-4f8e-a395-95a3700d963e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.357170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:36.362596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.363467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.363887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.364290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.365082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.367133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.377957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 803/1584] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/users => generated 289 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.378746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.391946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 782/1585] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.393552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.396356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14a269-ef06-4e1a-9cf3-df194d746408 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.402520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.403586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.403930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.404290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.405087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.407278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef764c1-1637-4c75-b73e-672dfc9beb6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.407764 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.417461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.418390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.418773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.420764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.422000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.424049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.437978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 804/1586] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/users => generated 291 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:36.437978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.439919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 783/1587] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.440914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.449958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1caf05b-4e4d-41e1-a38c-3c301b79654b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.452089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2034e-83c0-4698-86b3-2672c3d6d894 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.455939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.456820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.457244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.457393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.457711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.458027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.458316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.458594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.458846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.459313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.460749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.461214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.474369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 784/1588] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.475151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.480384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 805/1589] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.481387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.486950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c1e1c-4de5-4923-a313-14bf359f0d1b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.491912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.492690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.492972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.493246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.493961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.495467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb8672-729d-4109-a295-c501e787950e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.495602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.501171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.502191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.502579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.502958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.512264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.512493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 785/1590] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.513403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.514793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.524356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91d3f6-207e-47bb-ac4e-651c4681a2aa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.530025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 806/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.531011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.533586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.534303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.534593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.535351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.536169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.538309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.547664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fee89-97ea-4d4b-b247-dc1375091074 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.553677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.554107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 786/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.554570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.554953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.555095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.555571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.557368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.559444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.569725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f45de8-f6fe-4634-9615-52e16d3c3ff1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.580296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.580783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 807/1593] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.580912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.580912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.580912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.581190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.581291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.581291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.591167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54279610-cbd4-4b79-9851-9eac319886c4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.596511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 787/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.597608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.600100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.601143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.601534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.601908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] PATH_INFO: `/v3/projects/89ba51f0146e47ad9405899306d304db/users/ccfe3bb99949474a86a74b4683f843be/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.617484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6de5fb-45e0-42c8-bae8-0dcc8fc95b1b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.624454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.625423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.626007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.626168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ba51f0146e47ad9405899306d304db, user_id=ccfe3bb99949474a86a74b4683f843be, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.626502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] PATH_INFO: `/v3/projects/7a22ae4c671d4c828ea95651dd7ec9cd/users/f43bfa3b0a2a4fa28ef2e80295fed484/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.629515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.647143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a22ae4c671d4c828ea95651dd7ec9cd, user_id=f43bfa3b0a2a4fa28ef2e80295fed484, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.648734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:44:36.648734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:44:36.648734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:44:36.651107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.656393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 808/1595] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:36 2022] PUT /identity/v3/projects/89ba51f0146e47ad9405899306d304db/users/ccfe3bb99949474a86a74b4683f843be/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:36.657841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.678411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 788/1596] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:36 2022] PUT /identity/v3/projects/7a22ae4c671d4c828ea95651dd7ec9cd/users/f43bfa3b0a2a4fa28ef2e80295fed484/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:36.680496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.747997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:36.764943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8f736-4027-492b-a9f0-fdeba857d89d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.800847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732e3d-c7b8-4fdd-82c1-03be97a2f8b9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.811888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.813057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.813446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.813772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.814472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.816537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.830054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 789/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.834251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.846384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fe51-00ff-4714-a243-e59b46dc8f96 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.852735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.854016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.854322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.854859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.855450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.855583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.856212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.856535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.856821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.857151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.858175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.860055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.870157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 809/1598] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.871153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.878990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 790/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.882101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:36.888443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e72866-d2a8-4bf1-8ad7-d9a7d0b151d5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.895496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793dead-7302-4c05-a9fe-eedbfafac91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.896194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793dead-7302-4c05-a9fe-eedbfafac91a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.896538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793dead-7302-4c05-a9fe-eedbfafac91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.896822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793dead-7302-4c05-a9fe-eedbfafac91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.897898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793dead-7302-4c05-a9fe-eedbfafac91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.899601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793dead-7302-4c05-a9fe-eedbfafac91a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.901703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19085-812e-4053-8445-ece865d97623 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:36.914920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5b94a-2cb3-46b0-88b6-0a35157d7d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:36.914920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b94a-2cb3-46b0-88b6-0a35157d7d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.914920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b94a-2cb3-46b0-88b6-0a35157d7d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.914920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b94a-2cb3-46b0-88b6-0a35157d7d27 admin admin] PATH_INFO: `/v3/projects/87a88010e32e4f3297249813b5896ec8/users/c6ac1b362e014cf794564203566956e2/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.928026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 810/1600] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:36.928026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.928808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.929113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:36.941001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5b94a-2cb3-46b0-88b6-0a35157d7d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a88010e32e4f3297249813b5896ec8, user_id=c6ac1b362e014cf794564203566956e2, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:36.955790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5b94a-2cb3-46b0-88b6-0a35157d7d27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:36.959779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:36.969803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] MFA Rules not processed for user `ccfe3bb99949474a86a74b4683f843be`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:36.971857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:36.986698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 791/1601] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:36 2022] PUT /identity/v3/projects/87a88010e32e4f3297249813b5896ec8/users/c6ac1b362e014cf794564203566956e2/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:36.988219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:36.989775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:36.990074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.011716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:37.013361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.086034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 811/1602] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.091578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.100379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 792/1603] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.101256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.115317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984d67-f1bc-4dbe-8bd6-5a8bfeb6b7ca None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.118042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f20c6-e9e5-4ddf-b009-93e9c1ac2665 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.121846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.122996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.123581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.124030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] PATH_INFO: `/v3/projects/2007395425194750a87edd6c0da44e66/users/9958aaa8690c468da2acec47b2bb5f23/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.130150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.131838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.132235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.132652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] PATH_INFO: `/v3/projects/9ec8aa5d991a4fe5aa11b2dae79eb34b/users/eba7d8d3e3a645f8a9e36833d0f1bf1c/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.147326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2007395425194750a87edd6c0da44e66, user_id=9958aaa8690c468da2acec47b2bb5f23, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.151077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.176966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 812/1604] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:37 2022] PUT /identity/v3/projects/2007395425194750a87edd6c0da44e66/users/9958aaa8690c468da2acec47b2bb5f23/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:37.176966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.184218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec8aa5d991a4fe5aa11b2dae79eb34b, user_id=eba7d8d3e3a645f8a9e36833d0f1bf1c, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.188357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.218014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 793/1605] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:37 2022] PUT /identity/v3/projects/9ec8aa5d991a4fe5aa11b2dae79eb34b/users/eba7d8d3e3a645f8a9e36833d0f1bf1c/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:37.218014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.295444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e1b05-c490-4c4a-8455-bfb4db87a828 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.300123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d675b3-0cfb-4de7-a71e-0fa0e0073aa6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.305212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384cc3c-51e6-4150-94a3-bf315fa73411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.306156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384cc3c-51e6-4150-94a3-bf315fa73411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.306462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384cc3c-51e6-4150-94a3-bf315fa73411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.306946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384cc3c-51e6-4150-94a3-bf315fa73411 admin admin] PATH_INFO: `/v3/projects/667a2ccd9c9045879b4b87e156997161/users/dd8055e46e6149d3897fb88af0b19d39/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.309354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.310289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.310574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.310854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] PATH_INFO: `/v3/projects/a124b343b9cb4e7b85a6c75d83c01589/users/7da0c375806b44c8ae6c8d622fc357d3/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.332884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384cc3c-51e6-4150-94a3-bf315fa73411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=667a2ccd9c9045879b4b87e156997161, user_id=dd8055e46e6149d3897fb88af0b19d39, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.335774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384cc3c-51e6-4150-94a3-bf315fa73411 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.343690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a124b343b9cb4e7b85a6c75d83c01589, user_id=7da0c375806b44c8ae6c8d622fc357d3, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.346854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.350439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:37.359602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 813/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:37 2022] PUT /identity/v3/projects/667a2ccd9c9045879b4b87e156997161/users/dd8055e46e6149d3897fb88af0b19d39/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:37.359602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.359602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.359602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.372369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 794/1607] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:37 2022] PUT /identity/v3/projects/a124b343b9cb4e7b85a6c75d83c01589/users/7da0c375806b44c8ae6c8d622fc357d3/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:37.372369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.389979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] MFA Rules not processed for user `c6ac1b362e014cf794564203566956e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:37.391335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.461949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e328a-8369-4015-9098-bc6fa85da06c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.475338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:37.479438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f410b-30bb-462e-8ef8-b6f32bcec079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.484638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f410b-30bb-462e-8ef8-b6f32bcec079 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.484927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f410b-30bb-462e-8ef8-b6f32bcec079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.485243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f410b-30bb-462e-8ef8-b6f32bcec079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.488052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f410b-30bb-462e-8ef8-b6f32bcec079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.488921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f410b-30bb-462e-8ef8-b6f32bcec079 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.534461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 814/1608] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.535722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.542507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.573047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 795/1609] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/projects => generated 363 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:37.574720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650f422-4caa-49c4-a760-c71de2cb4677 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.575187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650f422-4caa-49c4-a760-c71de2cb4677 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.575327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650f422-4caa-49c4-a760-c71de2cb4677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.622110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0650f422-4caa-49c4-a760-c71de2cb4677 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:37.625759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0650f422-4caa-49c4-a760-c71de2cb4677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.637643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a102ae-f2a0-4414-8a78-9ba1827f045e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.717365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 796/1610] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.721779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.722146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.722503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.725486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7ace-dec1-48d4-849b-d6d5f3fa9593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.726254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7ace-dec1-48d4-849b-d6d5f3fa9593 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.726549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7ace-dec1-48d4-849b-d6d5f3fa9593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.726835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7ace-dec1-48d4-849b-d6d5f3fa9593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.737737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb7ace-dec1-48d4-849b-d6d5f3fa9593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.739203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb7ace-dec1-48d4-849b-d6d5f3fa9593 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.752570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] MFA Rules not processed for user `eba7d8d3e3a645f8a9e36833d0f1bf1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:37.752786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 815/1611] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:37.754026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc578f-396f-4376-a824-50cfd47c00e9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.754316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc578f-396f-4376-a824-50cfd47c00e9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.754686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc578f-396f-4376-a824-50cfd47c00e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.755507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.774196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-05cc578f-396f-4376-a824-50cfd47c00e9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:37.776158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05cc578f-396f-4376-a824-50cfd47c00e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.786246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 816/1612] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.787578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.787871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.788153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.831421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] MFA Rules not processed for user `7da0c375806b44c8ae6c8d622fc357d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:37.832832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:37.854892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 797/1613] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.856345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.878314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c4de8-cc1c-4946-8add-04b4e3fb8505 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.887437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4fa4f-57e5-4e1f-9f45-e72ab41ebee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.888154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4fa4f-57e5-4e1f-9f45-e72ab41ebee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.888512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4fa4f-57e5-4e1f-9f45-e72ab41ebee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.889964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4fa4f-57e5-4e1f-9f45-e72ab41ebee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.890720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4fa4f-57e5-4e1f-9f45-e72ab41ebee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.892970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4fa4f-57e5-4e1f-9f45-e72ab41ebee0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.904526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:37.930911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 798/1614] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/users => generated 288 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:37.931278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6eaab-aeed-41c5-8945-a09faff60a1c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.933307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6eaab-aeed-41c5-8945-a09faff60a1c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.933619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6eaab-aeed-41c5-8945-a09faff60a1c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.935317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 799/1615] 10.222.0.16 () {54 vars in 961 bytes} [Thu May 26 13:44:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:37.936088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6eaab-aeed-41c5-8945-a09faff60a1c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.939627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 817/1616] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:37.939627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.949391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6e308-32ee-4ba8-a4be-4c9528728cd6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.954099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.954862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.955217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.955560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.958036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6eaab-aeed-41c5-8945-a09faff60a1c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:37.962089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:37.963379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:37.963664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:37.964211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:37.975616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.977009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:37.977332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.979665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:37.991041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 818/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:37.991904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:37.996155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 800/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:37.997237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.003926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362aee44-178c-4624-9cda-7256679f1690 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.010674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.011399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.011772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.012054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.012870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.013068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522220d1-c848-4677-8aaa-39e483b17efd service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.015170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.020834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28179167-f999-4f3b-829e-3adb8add3f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.021695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28179167-f999-4f3b-829e-3adb8add3f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.022091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28179167-f999-4f3b-829e-3adb8add3f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.022517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28179167-f999-4f3b-829e-3adb8add3f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.023918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28179167-f999-4f3b-829e-3adb8add3f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.027747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28179167-f999-4f3b-829e-3adb8add3f53 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.035991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:38.053102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 801/1619] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:38.054885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.055289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.055680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.059276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 819/1620] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:37 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:38.059276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.101826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:38.104235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:38.143278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d26e95-7a40-415b-bcae-3c786414cfae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.148953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.149612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.149906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.150343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.151618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.153742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.184870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 820/1621] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:38 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:38.185917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.255298 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 802/1622] 10.222.0.16 () {58 vars in 1058 bytes} [Thu May 26 13:44:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:38.256884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.280220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f7183-17a4-4e0c-985d-bfb323a97728 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.289672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f129a7-7ee9-4bb8-9488-f14032967c6b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.293415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.294467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.294864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.295282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.296341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.299387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.301313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.304934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.305141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.305416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] PATH_INFO: `/v3/projects/b055bade54d04efebc2155a071837b7b/users/890a6f879e3e492dabf3d4a2aa45fcd6/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.327339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b055bade54d04efebc2155a071837b7b, user_id=890a6f879e3e492dabf3d4a2aa45fcd6, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.330149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.341525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 803/1623] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:38 2022] POST /identity/v3/users => generated 288 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:38.341525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.356558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 821/1624] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:38 2022] PUT /identity/v3/projects/b055bade54d04efebc2155a071837b7b/users/890a6f879e3e492dabf3d4a2aa45fcd6/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:38.357505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.370036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e1e8c-7bb8-43f9-9d30-ae6a3291fe6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.517449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:38.522809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.523494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.524475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.524475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.524940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.527987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.539587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a377a-db74-4655-93c0-53330541d7d8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.548983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.549572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.550655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.550655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.589674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 804/1625] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:38 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:38.590716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.616355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27420bfd-a856-4aa0-a8dc-60d20325d461 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.647908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.648713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.649237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.649530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.650979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.654236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.674625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 805/1626] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:38.675502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.676812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.686717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.689246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 822/1627] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:38.690329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.700708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72f7de-7cd3-4bac-81ef-3fcb7a5323a7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.703048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8852d7-ff5e-41cd-a5ca-36eb732b9f9c service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.708213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.708958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.709242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.709719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.710531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.714596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.718339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.720337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.721264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.722609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.725263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.726849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 823/1628] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:38.728348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.740101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.751247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec2dc4-b973-40c5-8118-64bff103d954 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.763136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 806/1629] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:38.763831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.766914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.768073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.768553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.768998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] PATH_INFO: `/v3/projects/787717cbcfb2471287687052daab71d5/users/81ddf0a79b994103ba1cb7e23c43ee6f/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.788834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bce4b-c1a6-4134-b094-38401fa75cde admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:38.799525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:38.800816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:38.801299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:38.802090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787717cbcfb2471287687052daab71d5, user_id=81ddf0a79b994103ba1cb7e23c43ee6f, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.802469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] PATH_INFO: `/v3/projects/4b6885417b824a37a7beec06c9942bb0/users/1688ea1a38bb40ee91935065edcd3bb4/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:38.809401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.810868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:38.836575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 824/1630] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:38 2022] PUT /identity/v3/projects/787717cbcfb2471287687052daab71d5/users/81ddf0a79b994103ba1cb7e23c43ee6f/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:38.837449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:38.842772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6885417b824a37a7beec06c9942bb0, user_id=1688ea1a38bb40ee91935065edcd3bb4, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:38.847710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:38.878355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 807/1631] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:38 2022] PUT /identity/v3/projects/4b6885417b824a37a7beec06c9942bb0/users/1688ea1a38bb40ee91935065edcd3bb4/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:38.878355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:39.021077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17c314-7629-4549-b528-65af62a40512 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:39.025443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bd00f-41dd-4a29-a397-76cc0b56f4fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:39.034685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:39.036140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.036274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:39.036477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.036948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.037232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.037878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.038210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] PATH_INFO: `/v3/projects/b055bade54d04efebc2155a071837b7b/users/890a6f879e3e492dabf3d4a2aa45fcd6/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.041780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b055bade54d04efebc2155a071837b7b, user_id=890a6f879e3e492dabf3d4a2aa45fcd6, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:39.042294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:39.043963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:39.052937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:39.088466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 825/1632] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:38 2022] PUT /identity/v3/projects/b055bade54d04efebc2155a071837b7b/users/890a6f879e3e492dabf3d4a2aa45fcd6/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:39.088466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:39.088826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 808/1633] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:39.088826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:39.110368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616c45f-30a8-4d1e-b46a-5f80d55b489d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:39.273649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabfe3c-6d46-4884-b30f-4077a0109dd0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:39.294409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef61c5-566e-41d8-8498-3d38afb45253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:39.299136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef61c5-566e-41d8-8498-3d38afb45253 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.299469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef61c5-566e-41d8-8498-3d38afb45253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.299711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef61c5-566e-41d8-8498-3d38afb45253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.300618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef61c5-566e-41d8-8498-3d38afb45253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:39.302225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef61c5-566e-41d8-8498-3d38afb45253 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:39.305908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:39.325831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 826/1634] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:39.327341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5096fd-36ae-4fbd-bb70-f2371f8a3d54 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.327665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5096fd-36ae-4fbd-bb70-f2371f8a3d54 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.328352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5096fd-36ae-4fbd-bb70-f2371f8a3d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.345140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:39.346129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.346419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.346697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] PATH_INFO: `/v3/projects/787717cbcfb2471287687052daab71d5/users/81ddf0a79b994103ba1cb7e23c43ee6f/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.353509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=787717cbcfb2471287687052daab71d5, user_id=81ddf0a79b994103ba1cb7e23c43ee6f, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:39.357470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:39.390299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 809/1635] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:39 2022] PUT /identity/v3/projects/787717cbcfb2471287687052daab71d5/users/81ddf0a79b994103ba1cb7e23c43ee6f/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:39.391953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:39.417847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-de5096fd-36ae-4fbd-bb70-f2371f8a3d54 None None] MFA Rules not processed for user `890a6f879e3e492dabf3d4a2aa45fcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:39.423163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de5096fd-36ae-4fbd-bb70-f2371f8a3d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:39.570988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8a498-72e6-4203-a2ac-d9046be2e23e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:39.593861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fadc9-b2d2-4c93-a941-50eff3055b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:39.594848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fadc9-b2d2-4c93-a941-50eff3055b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.595137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fadc9-b2d2-4c93-a941-50eff3055b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.595467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fadc9-b2d2-4c93-a941-50eff3055b15 admin admin] PATH_INFO: `/v3/projects/4b6885417b824a37a7beec06c9942bb0/users/1688ea1a38bb40ee91935065edcd3bb4/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.605616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fadc9-b2d2-4c93-a941-50eff3055b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6885417b824a37a7beec06c9942bb0, user_id=1688ea1a38bb40ee91935065edcd3bb4, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:39.611157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fadc9-b2d2-4c93-a941-50eff3055b15 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:39.657528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 810/1636] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:39 2022] PUT /identity/v3/projects/4b6885417b824a37a7beec06c9942bb0/users/1688ea1a38bb40ee91935065edcd3bb4/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:39.670790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c54d6f-ebfd-4930-a8bf-aed269ad95f0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.672501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c54d6f-ebfd-4930-a8bf-aed269ad95f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.673549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c54d6f-ebfd-4930-a8bf-aed269ad95f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.722085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:39.744541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-56c54d6f-ebfd-4930-a8bf-aed269ad95f0 None None] MFA Rules not processed for user `81ddf0a79b994103ba1cb7e23c43ee6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:39.746172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56c54d6f-ebfd-4930-a8bf-aed269ad95f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:39.795616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 827/1637] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:39.795616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.795616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.795616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:39.913637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] MFA Rules not processed for user `1688ea1a38bb40ee91935065edcd3bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:39.914888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:39.955910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 811/1638] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:39.957522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673ad85-22c0-45bc-9709-12ea2b19ec0c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:39.958120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673ad85-22c0-45bc-9709-12ea2b19ec0c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:39.958279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673ad85-22c0-45bc-9709-12ea2b19ec0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.017230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3673ad85-22c0-45bc-9709-12ea2b19ec0c None None] MFA Rules not processed for user `f43bfa3b0a2a4fa28ef2e80295fed484`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:40.018613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3673ad85-22c0-45bc-9709-12ea2b19ec0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:40.150323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 828/1639] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:40.150323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:40.163064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 812/1640] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:40.169109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f862-cfcb-454c-b0f0-2a5bb9dd4ec0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.169678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f862-cfcb-454c-b0f0-2a5bb9dd4ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.169970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f862-cfcb-454c-b0f0-2a5bb9dd4ec0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.176882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 813/1641] 10.222.0.16 () {54 vars in 961 bytes} [Thu May 26 13:44:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:40.177793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17f862-cfcb-454c-b0f0-2a5bb9dd4ec0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:40.245906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d4f1a-8a34-4681-b8e4-c483b46a3ebe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:40.251607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3c8b5-c4d9-4548-b119-42f594b9286c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:40.253038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c8b5-c4d9-4548-b119-42f594b9286c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.253466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c8b5-c4d9-4548-b119-42f594b9286c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.253944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c8b5-c4d9-4548-b119-42f594b9286c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.282317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:40.323243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17f862-cfcb-454c-b0f0-2a5bb9dd4ec0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:40.332104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:40.332994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.333331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.334261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.342773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3c8b5-c4d9-4548-b119-42f594b9286c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:40.344551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3c8b5-c4d9-4548-b119-42f594b9286c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:40.359149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:40.359428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 829/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:40.361630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e335-3800-4336-aa81-07eec926f0fc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.362049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e335-3800-4336-aa81-07eec926f0fc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.362439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e335-3800-4336-aa81-07eec926f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.363841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:40.385185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d553e335-3800-4336-aa81-07eec926f0fc None None] MFA Rules not processed for user `f43bfa3b0a2a4fa28ef2e80295fed484`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:40.386746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d553e335-3800-4336-aa81-07eec926f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:40.399901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 830/1643] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:40.403971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 814/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:40.408009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.408332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.409656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.412510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:40.491484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:40.502857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:40.575149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:40.705753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159446-4b35-4540-8c9f-3593cd17d650 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:40.739894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:40.746695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.746695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.748749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.783399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:40.793990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:40.797579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 831/1645] 10.222.0.16 () {58 vars in 1058 bytes} [Thu May 26 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:40.800631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:40.822346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 815/1646] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:44:40.823868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f2bc7-86ac-4e1b-920f-b32b76c21475 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:40.824616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:40.834649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c8487-74fb-4649-afbb-99849934475a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:40.835341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c8487-74fb-4649-afbb-99849934475a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.835632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c8487-74fb-4649-afbb-99849934475a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.835905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c8487-74fb-4649-afbb-99849934475a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.844075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e2148-ece6-4621-b29b-e10247ce070d service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:40.855025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b580d5-3456-4935-87f4-46f557861e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:40.855702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b580d5-3456-4935-87f4-46f557861e39 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.855988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b580d5-3456-4935-87f4-46f557861e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.856263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b580d5-3456-4935-87f4-46f557861e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.859340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c8487-74fb-4649-afbb-99849934475a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:40.861166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c8487-74fb-4649-afbb-99849934475a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:40.892503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 832/1647] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:40.896380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6aa00-b97b-46df-a8e1-dd6012828820 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.897477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6aa00-b97b-46df-a8e1-dd6012828820 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.898034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6aa00-b97b-46df-a8e1-dd6012828820 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.902547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 833/1648] 10.222.0.16 () {54 vars in 912 bytes} [Thu May 26 13:44:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:40.912878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:40.912878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:40.912878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:40.970809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:40.973968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:41.011531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 834/1649] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:41.013729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:41.045460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92686dd2-7f5a-40c3-8639-2392bb9aeb45 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:41.049439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b580d5-3456-4935-87f4-46f557861e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:41.059670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b580d5-3456-4935-87f4-46f557861e39 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:41.063904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:41.065271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:41.065574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:41.067877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:41.088784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 816/1650] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:41.090622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82c157-44af-43e4-b335-19844d2c0796 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:41.091496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82c157-44af-43e4-b335-19844d2c0796 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:41.092398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82c157-44af-43e4-b335-19844d2c0796 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:41.101648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 817/1651] 10.222.0.16 () {54 vars in 912 bytes} [Thu May 26 13:44:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:41.109145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:41.110285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:41.110285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:41.149481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:41.150683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:41.187978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 818/1652] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:44:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:41.202532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:41.203966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:41.257399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:41.264906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:41.289040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 835/1653] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:41.349074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3e55c-6bdb-4f11-853d-9c80da3ddb00 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:41.369587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:41.370640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:41.371109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:41.371547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:41.390122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:41.392007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:41.420205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 819/1654] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:44:41.600871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:41.613337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075a416-426a-4581-b9aa-32f0c53bf90a service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:41.625470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:41.626378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:41.626771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:41.627159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:41.642260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:41.644799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:41.673808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 836/1655] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:41.856559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:41.867535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd14c-e346-43ba-badb-73e056013cef service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:41.871869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241de09-467b-4f2e-8c06-d09c56961457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:41.872774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241de09-467b-4f2e-8c06-d09c56961457 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:41.873168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241de09-467b-4f2e-8c06-d09c56961457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:41.873563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241de09-467b-4f2e-8c06-d09c56961457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:41.953582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:41.973832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241de09-467b-4f2e-8c06-d09c56961457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:41.975919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241de09-467b-4f2e-8c06-d09c56961457 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:42.014050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 820/1656] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:42.525660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:42.541237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:42.672824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:42.673128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:42.673468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:42.703345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb4ab9-ca60-4b2d-a121-2af2a13ecdcc service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:42.708319 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:42.710129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:42.710409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:42.710698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] PATH_INFO: `/v3/users/c6ac1b362e014cf794564203566956e2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:42.713188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ac1b362e014cf794564203566956e2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:42.717974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:42.726813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] MFA Rules not processed for user `e13f5cad16824554a1fb32ed9a915ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:42.737940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:42.866681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ac1b362e014cf794564203566956e2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:42.868132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ac1b362e014cf794564203566956e2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:42.882705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ac1b362e014cf794564203566956e2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:42.885007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ac1b362e014cf794564203566956e2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:42.899701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b94ac0ec-3f13-441b-a6e1-07cd45369029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ac1b362e014cf794564203566956e2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:42.947122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 821/1657] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:44:42.973997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:42.977066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:42.977727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 837/1658] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:42 2022] DELETE /identity/v3/users/c6ac1b362e014cf794564203566956e2 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:42.982644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:42.983439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:42.983967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:43.012457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:43.015464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e28e48-fad4-429b-b0f0-77ddc8593c8f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:43.030408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:43.185150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609adbe-b24e-4cf1-b952-4594a706d6c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:43.185876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609adbe-b24e-4cf1-b952-4594a706d6c9 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:43.186119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609adbe-b24e-4cf1-b952-4594a706d6c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:43.186399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609adbe-b24e-4cf1-b952-4594a706d6c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:43.231496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 838/1659] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:44:43.245358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:43.246797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:43.313994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:43.352038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609adbe-b24e-4cf1-b952-4594a706d6c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:43.356033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609adbe-b24e-4cf1-b952-4594a706d6c9 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:43.374716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 822/1660] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:43.376883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0096d-3768-4aef-bf32-4f0a2f3fb9d1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:43.377347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0096d-3768-4aef-bf32-4f0a2f3fb9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:43.377800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0096d-3768-4aef-bf32-4f0a2f3fb9d1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:43.386930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 823/1661] 10.222.0.16 () {54 vars in 912 bytes} [Thu May 26 13:44:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:43.388474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:43.388841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:43.389192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:43.413937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fa7e9-d4ad-4b7a-b3f0-9e74a756a2ac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:43.431092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:43.431882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:43.432359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:43.432643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:43.450019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:43.457889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:43.461627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:43.497443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 839/1662] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:43.507079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] MFA Rules not processed for user `6f44d10be1654d17990966ae64d43ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:43.511048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:43.544891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 824/1663] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:44:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:43.621034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:43.622716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:43.644936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd348df-5e1f-4b6e-b993-2aedb9320301 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:43.669132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd511e2-132b-49ec-8402-60bde8f42310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:43.671302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd511e2-132b-49ec-8402-60bde8f42310 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:43.674051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd511e2-132b-49ec-8402-60bde8f42310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:43.674366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd511e2-132b-49ec-8402-60bde8f42310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:43.705085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd511e2-132b-49ec-8402-60bde8f42310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:43.708339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd511e2-132b-49ec-8402-60bde8f42310 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:43.746188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 840/1664] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:44.127134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:44.129227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:44.161153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70aa6-19e6-4acd-97e8-ce8fc2d12b31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:44.167402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:44.168643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:44.169145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:44.169613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] PATH_INFO: `/v3/projects/87a88010e32e4f3297249813b5896ec8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:44.171708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a88010e32e4f3297249813b5896ec8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:44.174599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:44.287117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a88010e32e4f3297249813b5896ec8', 'request_id': 'req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:44.288206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a88010e32e4f3297249813b5896ec8', 'request_id': 'req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:44.302770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a88010e32e4f3297249813b5896ec8', 'request_id': 'req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:44.352238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] The token cache is being invalidate because project 87a88010e32e4f3297249813b5896ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:44.352570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:44.354599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e58e608e-92bd-4092-a48a-b92c8e9d5bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:44.359751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 825/1665] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:44 2022] DELETE /identity/v3/projects/87a88010e32e4f3297249813b5896ec8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:44.363757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:44.364894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:44.365780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:44.398363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:44.399776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:44.555240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 841/1666] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:44.565377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:44.565664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:44.565948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:44.614472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:44.616025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:44.636362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 826/1667] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:44.640257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:44.656750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834606d-1f25-411f-8aa3-cd95b3b5e888 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:44.670895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:44.673422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:44.673422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:44.673952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:44.678886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:44.679216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:44.684739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:44.730196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 842/1668] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:44.733508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:44.747988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fe3bb-9844-4524-8d1e-6cbda60d2679 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:44.757205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:44.758242 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:44.758999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:44.759578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:44.760635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:44.763444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:44.819262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 827/1669] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:44 2022] POST /identity/v3/projects => generated 403 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:44.825901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:44.839594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27d04-7a94-4a37-bfc1-0c026c59e599 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:44.998510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:45.000647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.002503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.003704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.005449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:45.011008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:45.058700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 843/1670] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:44 2022] POST /identity/v3/users => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:44:45.063476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:45.078891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a68549-adbb-4fc8-be5d-dc6587e6ad65 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:45.097374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:45.098075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.098357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.098631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.099366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:45.110714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:45.173906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 828/1671] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:44:45.175777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:45.195162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2744c-6263-49ac-a847-df3f939ab6aa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:45.202946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee486fb-3293-4b8c-b4ef-3ae5097a7bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:45.203836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee486fb-3293-4b8c-b4ef-3ae5097a7bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.204232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee486fb-3293-4b8c-b4ef-3ae5097a7bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.204786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee486fb-3293-4b8c-b4ef-3ae5097a7bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.205513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee486fb-3293-4b8c-b4ef-3ae5097a7bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:45.207264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee486fb-3293-4b8c-b4ef-3ae5097a7bd8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:45.255267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 844/1672] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:45.258526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:45.283140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcb492-bfaa-4892-b72b-9dad9318e9c9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:45.296813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd67d54-fe6b-411b-adc9-64c6fc300345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:45.299226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67d54-fe6b-411b-adc9-64c6fc300345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.299983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67d54-fe6b-411b-adc9-64c6fc300345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.301014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67d54-fe6b-411b-adc9-64c6fc300345 admin admin] PATH_INFO: `/v3/projects/ca2fc36fc2df4c2e927997ea5b196b5c/users/8cb6207a5aa043ee91638abe3a156a04/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.350843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd67d54-fe6b-411b-adc9-64c6fc300345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2fc36fc2df4c2e927997ea5b196b5c, user_id=8cb6207a5aa043ee91638abe3a156a04, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:45.354600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd67d54-fe6b-411b-adc9-64c6fc300345 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:45.409603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 829/1673] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:45 2022] PUT /identity/v3/projects/ca2fc36fc2df4c2e927997ea5b196b5c/users/8cb6207a5aa043ee91638abe3a156a04/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:45.413000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.413723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.414043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.457789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:45.461848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:45.530224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9892e-0173-448e-ac16-a4f3aaa6265f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.530853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9892e-0173-448e-ac16-a4f3aaa6265f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.531148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9892e-0173-448e-ac16-a4f3aaa6265f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.532946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 830/1674] 10.222.0.16 () {54 vars in 960 bytes} [Thu May 26 13:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 13:44:45.538417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:45.539729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.540028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.540579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.609437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:45.610887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:45.720033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 845/1675] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:44:45.743735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:45.746222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:45.858966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 831/1676] 10.222.0.16 () {58 vars in 1057 bytes} [Thu May 26 13:44:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:44:45.873265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:45.886565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406bd8d-91eb-4a17-adf2-d2112fa8e9ae None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:45.903737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:45.905406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.906957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.907249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:45.912491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:45.950280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:45.966408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:45.968034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:45.979456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc768-fe40-43d7-8d0e-6dbebd19acb8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:45.985347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddd591-181c-4e3b-af75-a2838456ff81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:45.986516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddd591-181c-4e3b-af75-a2838456ff81 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:45.987112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddd591-181c-4e3b-af75-a2838456ff81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:45.987606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddd591-181c-4e3b-af75-a2838456ff81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:46.002007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddd591-181c-4e3b-af75-a2838456ff81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:46.005082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddd591-181c-4e3b-af75-a2838456ff81 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:46.017500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 832/1677] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:44:46.039246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 846/1678] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:44:46.204271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:46.205971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:46.237075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afc853-ac05-4207-8f66-64055acf3b7d service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:46.267513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:46.268625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:46.269018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:46.269427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] PATH_INFO: `/v3/users/eba7d8d3e3a645f8a9e36833d0f1bf1c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:46.276939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba7d8d3e3a645f8a9e36833d0f1bf1c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:46.284353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:44:46.284353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 26 13:44:46.284353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: May 26 13:44:46.287301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:46.497351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba7d8d3e3a645f8a9e36833d0f1bf1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:46.560078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba7d8d3e3a645f8a9e36833d0f1bf1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:46.575679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba7d8d3e3a645f8a9e36833d0f1bf1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:46.594347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba7d8d3e3a645f8a9e36833d0f1bf1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:46.595066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba7d8d3e3a645f8a9e36833d0f1bf1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:46.648832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 833/1679] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:46 2022] DELETE /identity/v3/users/eba7d8d3e3a645f8a9e36833d0f1bf1c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:46.653347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cdba2-157f-4ca9-b3d5-7750a0ec5a8a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:46.653652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cdba2-157f-4ca9-b3d5-7750a0ec5a8a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:46.653947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cdba2-157f-4ca9-b3d5-7750a0ec5a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:46.674038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a90cdba2-157f-4ca9-b3d5-7750a0ec5a8a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:46.675652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a90cdba2-157f-4ca9-b3d5-7750a0ec5a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:46.769798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 847/1680] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:46.817678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:46.819872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:47.010330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291022a-da74-4c19-8302-9bb64f8e1fdc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:47.016686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:47.017364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:47.017655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:47.017987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:47.036597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:47.038169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:47.053226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 834/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:47.599896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:47.600198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:47.600512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:47.655494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] MFA Rules not processed for user `8cb6207a5aa043ee91638abe3a156a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:47.656771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:47.727368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:47.729299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:47.751674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995abdd4-7189-4ffc-bc1e-bb8bcf4bef9a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:47.764161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:47.767010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:47.767304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:47.767588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] PATH_INFO: `/v3/projects/9ec8aa5d991a4fe5aa11b2dae79eb34b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:47.769978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec8aa5d991a4fe5aa11b2dae79eb34b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:47.771992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:47.778566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:47.802400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 848/1682] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:47.807291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:47.809308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:47.841600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfe187-cd17-43aa-86e1-1b3685eec93c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:47.856236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:47.856949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:47.857242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:47.858503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:47.859217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:47.861354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:47.881279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec8aa5d991a4fe5aa11b2dae79eb34b', 'request_id': 'req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:47.894951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec8aa5d991a4fe5aa11b2dae79eb34b', 'request_id': 'req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:47.897576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 849/1683] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:47 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:47.902570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:47.907755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:47.935020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec8aa5d991a4fe5aa11b2dae79eb34b', 'request_id': 'req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:47.957115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] The token cache is being invalidate because project 9ec8aa5d991a4fe5aa11b2dae79eb34b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:47.957601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:47.958510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:47.960423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 835/1684] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:47 2022] DELETE /identity/v3/projects/9ec8aa5d991a4fe5aa11b2dae79eb34b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:47.964713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:47.966362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:48.007197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01da2-e64f-4161-99ed-d25fe81723ee admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:48.012768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:48.013707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:48.014068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:48.014420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:48.015695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:48.017874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:48.048994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 850/1685] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:47 2022] POST /identity/v3/users => generated 308 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:48.050427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:48.056119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:48.079947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b4bc-9cc7-4edf-a0e8-6031da5a150c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:48.086172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:48.088841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:48.088841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:48.089412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:48.090419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:48.092122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:48.109827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 851/1686] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:48.114568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:48.127586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2325bf0-7d2b-40bd-b0a4-f344f0b9e8ae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:48.135898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:48.137395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:48.137889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:48.138327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] PATH_INFO: `/v3/projects/4d1e9baec0a64d06858f2f49ad3b6eab/users/3030e96412994189b7c99532bdeda818/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:48.154848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623eb51-c70c-42a9-a3ef-bdae98d46f5b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:48.157272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1e9baec0a64d06858f2f49ad3b6eab, user_id=3030e96412994189b7c99532bdeda818, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:48.161189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:48.164231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:48.170944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:48.171250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:48.172433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] PATH_INFO: `/v3/users/81ddf0a79b994103ba1cb7e23c43ee6f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:48.177546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ddf0a79b994103ba1cb7e23c43ee6f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:48.181297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:48.209955 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 852/1687] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:48 2022] PUT /identity/v3/projects/4d1e9baec0a64d06858f2f49ad3b6eab/users/3030e96412994189b7c99532bdeda818/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:48.211058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:48.270835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ddf0a79b994103ba1cb7e23c43ee6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:48.295795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ddf0a79b994103ba1cb7e23c43ee6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:48.296607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ddf0a79b994103ba1cb7e23c43ee6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:48.311306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ddf0a79b994103ba1cb7e23c43ee6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:48.312328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ddf0a79b994103ba1cb7e23c43ee6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:48.332934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 836/1688] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:47 2022] DELETE /identity/v3/users/81ddf0a79b994103ba1cb7e23c43ee6f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:48.339732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8b08c-de40-434b-b97b-55de50e82391 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:48.390872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:48.444497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:48.445558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:48.446071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:48.446653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:48.448064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:48.452956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:48.471601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 853/1689] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:48.473303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:48.475464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:48.501413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f718994-b831-46f5-8194-ae9339c3b262 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:48.513872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bd99a-ce8f-4a25-ab16-82a61893bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:48.518216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd99a-ce8f-4a25-ab16-82a61893bda3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:48.518769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd99a-ce8f-4a25-ab16-82a61893bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:48.519252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd99a-ce8f-4a25-ab16-82a61893bda3 admin admin] PATH_INFO: `/v3/projects/4d1e9baec0a64d06858f2f49ad3b6eab/users/3030e96412994189b7c99532bdeda818/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:48.523195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bd99a-ce8f-4a25-ab16-82a61893bda3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1e9baec0a64d06858f2f49ad3b6eab, user_id=3030e96412994189b7c99532bdeda818, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:48.528656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bd99a-ce8f-4a25-ab16-82a61893bda3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:48.546059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 837/1690] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:48 2022] PUT /identity/v3/projects/4d1e9baec0a64d06858f2f49ad3b6eab/users/3030e96412994189b7c99532bdeda818/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:49.615709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:49.618237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:49.645411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b012ed-185a-4956-8803-27e0a3af74f1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:49.666396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdec0b-c486-4b52-a51e-15345603e48e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:49.667126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdec0b-c486-4b52-a51e-15345603e48e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:49.667126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdec0b-c486-4b52-a51e-15345603e48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:49.745974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-24bdec0b-c486-4b52-a51e-15345603e48e None None] MFA Rules not processed for user `3030e96412994189b7c99532bdeda818`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:49.752803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-24bdec0b-c486-4b52-a51e-15345603e48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:49.821328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:49.822290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:49.822588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:49.822875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] PATH_INFO: `/v3/projects/787717cbcfb2471287687052daab71d5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:49.826014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787717cbcfb2471287687052daab71d5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:49.826994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 13:44:49.826994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 26 13:44:49.826994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: May 26 13:44:49.835831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:49.851913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:49.918360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 838/1691] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:49.963176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787717cbcfb2471287687052daab71d5', 'request_id': 'req-b883f810-374b-4b0b-9d71-f064ba345d74'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:49.974505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787717cbcfb2471287687052daab71d5', 'request_id': 'req-b883f810-374b-4b0b-9d71-f064ba345d74'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:49.996935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787717cbcfb2471287687052daab71d5', 'request_id': 'req-b883f810-374b-4b0b-9d71-f064ba345d74'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:50.014470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] The token cache is being invalidate because project 787717cbcfb2471287687052daab71d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:50.014952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:50.015789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b883f810-374b-4b0b-9d71-f064ba345d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:50.017895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 854/1692] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:49 2022] DELETE /identity/v3/projects/787717cbcfb2471287687052daab71d5 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:50.023359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.023743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.024122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.069402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:50.073004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:50.221153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 839/1693] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:50.230721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.235896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.239035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.304323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:50.306459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:50.315914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 855/1694] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:50.319861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:50.342719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c464a-cca3-44c5-ad91-5172b282a29c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:50.350465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:50.351427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.351906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.353019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.354312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:50.354819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:50.367179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:50.408343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 840/1695] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:50.412293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:50.434688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd235eb4-60aa-45c1-b452-0a217197cc42 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:50.450306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:50.451016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.451428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.451709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.452915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:50.454858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:50.535101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 856/1696] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:50 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:50.539632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:50.554034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9b823-2e84-4efa-9183-dd7ea6d67a48 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:50.703146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:50.703832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.704107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.704416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.705136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:50.707144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:50.762785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 841/1697] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:50 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:50.766542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:50.785117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1d54-8882-4922-902b-be682fd1dc62 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:50.799886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:50.800867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.801263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.801651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.802538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:50.804396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:50.829819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 857/1698] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:50.833810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:50.846811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792fcc4-0ee9-4908-9bdd-bd20ed80de3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:50.852135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf8d25-7fea-49c8-9d86-928e7c534afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:50.853243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf8d25-7fea-49c8-9d86-928e7c534afc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.853718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf8d25-7fea-49c8-9d86-928e7c534afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.854098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf8d25-7fea-49c8-9d86-928e7c534afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.855152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf8d25-7fea-49c8-9d86-928e7c534afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:50.857708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf8d25-7fea-49c8-9d86-928e7c534afc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:50.869882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 842/1699] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:50.873678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:50.887556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31607453-7c77-45f4-bb93-029883a47f41 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:50.911927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:50.912900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:50.913183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:50.919565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] PATH_INFO: `/v3/projects/42f1fb9950244410993a0e2101f4c56d/users/71452ea1927b4357a147d36a52a88d2f/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:50.988001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f1fb9950244410993a0e2101f4c56d, user_id=71452ea1927b4357a147d36a52a88d2f, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:50.991190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:51.007056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:51.033868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 858/1700] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:50 2022] PUT /identity/v3/projects/42f1fb9950244410993a0e2101f4c56d/users/71452ea1927b4357a147d36a52a88d2f/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:51.044796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124cfc-4f88-433f-adcd-172d47950386 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.045092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124cfc-4f88-433f-adcd-172d47950386 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.045362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1124cfc-4f88-433f-adcd-172d47950386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.110244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f1124cfc-4f88-433f-adcd-172d47950386 None None] MFA Rules not processed for user `71452ea1927b4357a147d36a52a88d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:51.111861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1124cfc-4f88-433f-adcd-172d47950386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:51.248515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:51.289455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 843/1701] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:51.293799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:51.435683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1deb38-4954-4d2f-a74c-d4ed2eee48de admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:51.445920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:51.447325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.447686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.448015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.449874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:51.452233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:51.463828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214897f-9daa-455c-adf1-d0ae6646bdb9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.464110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214897f-9daa-455c-adf1-d0ae6646bdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.464420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214897f-9daa-455c-adf1-d0ae6646bdb9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.465936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 844/1702] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:51.498001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.498497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.499060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.521359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 859/1703] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:44:51 2022] POST /identity/v3/projects => generated 389 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:44:51.548379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:51.569649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:51.569790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:51.754910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfde50-a6d1-4188-bec6-e32b8c4f9d18 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:51.779255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 845/1704] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:44:51.786583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacbec4-52f3-42cf-8169-5ed134a5f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:51.787297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacbec4-52f3-42cf-8169-5ed134a5f461 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.787686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacbec4-52f3-42cf-8169-5ed134a5f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.788110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacbec4-52f3-42cf-8169-5ed134a5f461 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.791388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacbec4-52f3-42cf-8169-5ed134a5f461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:51.795012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacbec4-52f3-42cf-8169-5ed134a5f461 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:51.801017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:51.802633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:51.825772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f2eb2-c781-4a13-bab6-ba003c821110 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:51.847651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:51.848656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.848925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.849323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.857221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 860/1705] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:44:51 2022] POST /identity/v3/users => generated 301 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:44:51.858788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.859091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.859386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.869830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:51.872327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:51.896429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 846/1706] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:44:51.897408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:51.927623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604f407-3a8c-4bc7-8778-e32283fe59d2 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:51.941575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] MFA Rules not processed for user `e13f5cad16824554a1fb32ed9a915ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:51.943122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:51.953398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:51.953649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:51.953934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:51.954216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:51.955003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:51.956695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:51.995595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 847/1707] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:52.001406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:52.026593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb5d29-a609-4318-bb6c-aefecffc16f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:52.037396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:52.040117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:52.040626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:52.042478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] PATH_INFO: `/v3/projects/ffee65c6a4d94585896ea903da1ab723/users/af5d230fb52b439cab8b0b3f20a3ac7f/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:52.086450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffee65c6a4d94585896ea903da1ab723, user_id=af5d230fb52b439cab8b0b3f20a3ac7f, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:52.090380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:52.091522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 861/1708] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:52.154335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 848/1709] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:52 2022] PUT /identity/v3/projects/ffee65c6a4d94585896ea903da1ab723/users/af5d230fb52b439cab8b0b3f20a3ac7f/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:52.155680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:52.179936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fd8f8-0554-4ae3-9e74-110bc3398a54 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:52.226751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:52.295999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5dcab-49fd-4e31-ba5d-3e5bb9557ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:52.296960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5dcab-49fd-4e31-ba5d-3e5bb9557ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:52.297494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5dcab-49fd-4e31-ba5d-3e5bb9557ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:52.297907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5dcab-49fd-4e31-ba5d-3e5bb9557ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:52.298934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5dcab-49fd-4e31-ba5d-3e5bb9557ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:52.300952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5dcab-49fd-4e31-ba5d-3e5bb9557ae4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:52.320189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 862/1710] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:44:52.322755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:52.354407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d4adb-f044-4659-b9a4-e3391e2d5273 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:52.370939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:52.372245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:52.372785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:52.373283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] PATH_INFO: `/v3/projects/ffee65c6a4d94585896ea903da1ab723/users/af5d230fb52b439cab8b0b3f20a3ac7f/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:52.375676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffee65c6a4d94585896ea903da1ab723, user_id=af5d230fb52b439cab8b0b3f20a3ac7f, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:52.380215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:52.396960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 849/1711] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:44:52 2022] PUT /identity/v3/projects/ffee65c6a4d94585896ea903da1ab723/users/af5d230fb52b439cab8b0b3f20a3ac7f/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:52.408723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:52.409014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:52.409300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:52.446029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] MFA Rules not processed for user `af5d230fb52b439cab8b0b3f20a3ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:52.448964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:52.828736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 863/1712] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:52.846656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:52.848738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:52.987566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abcac9-ec9c-43a3-b128-7aa15271821d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:52.994012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076eec-1633-4ae6-97b5-11716a8797a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:52.994856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076eec-1633-4ae6-97b5-11716a8797a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:52.995260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076eec-1633-4ae6-97b5-11716a8797a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:52.995644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076eec-1633-4ae6-97b5-11716a8797a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:53.015708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076eec-1633-4ae6-97b5-11716a8797a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:53.018448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076eec-1633-4ae6-97b5-11716a8797a9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:53.032268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 850/1713] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:53.215708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:53.309773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb0cb3-7435-4e75-abf8-4325f33f7368 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:53.316517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:53.322043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:53.322724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:53.323480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] PATH_INFO: `/v3/users/71452ea1927b4357a147d36a52a88d2f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:53.325737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71452ea1927b4357a147d36a52a88d2f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:53.328652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:53.385526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:53.392273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71452ea1927b4357a147d36a52a88d2f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:53.393054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71452ea1927b4357a147d36a52a88d2f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:53.402009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71452ea1927b4357a147d36a52a88d2f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:53.402787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71452ea1927b4357a147d36a52a88d2f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:53.412265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71452ea1927b4357a147d36a52a88d2f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:53.459892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 864/1714] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:53 2022] DELETE /identity/v3/users/71452ea1927b4357a147d36a52a88d2f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:53.465877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8cb0e-0b16-480e-abee-cd1b57b9a869 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:53.466179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8cb0e-0b16-480e-abee-cd1b57b9a869 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:53.466480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8cb0e-0b16-480e-abee-cd1b57b9a869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:53.487511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6be8cb0e-0b16-480e-abee-cd1b57b9a869 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:53.488543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6be8cb0e-0b16-480e-abee-cd1b57b9a869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:53.500834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:53.605341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 851/1715] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:53.636917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:53.642280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:53.794687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a539a-b0e0-4940-be1f-7081b58a7a8a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:53.814540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac936c-7c70-41dd-969b-35233bfd14b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:53.815421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac936c-7c70-41dd-969b-35233bfd14b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:53.816500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac936c-7c70-41dd-969b-35233bfd14b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:53.816798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac936c-7c70-41dd-969b-35233bfd14b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:53.835323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac936c-7c70-41dd-969b-35233bfd14b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:53.837173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac936c-7c70-41dd-969b-35233bfd14b6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:53.854155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 865/1716] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:54.062586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ec28d-7942-49f3-9a3e-9cad355ca561 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:54.063084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ec28d-7942-49f3-9a3e-9cad355ca561 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:54.063084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ec28d-7942-49f3-9a3e-9cad355ca561 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:54.064510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 852/1717] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:44:54.182894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:54.183197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:54.183494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:54.203275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] MFA Rules not processed for user `f14da5405ccf4fdfaf91b1ec52ad3801`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:54.204808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:54.277097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ec28d-7942-49f3-9a3e-9cad355ca561 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:54.280507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e08ec28d-7942-49f3-9a3e-9cad355ca561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:54.310475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ec28d-7942-49f3-9a3e-9cad355ca561 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:54.319381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:54.321312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:54.331327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:54.331849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] PATH_INFO: `/v3/users/8cb6207a5aa043ee91638abe3a156a04` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:54.334757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb6207a5aa043ee91638abe3a156a04)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:54.340078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:54.374765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 866/1718] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:44:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:54.399992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:54.402923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:54.434124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32239a33-ffd6-432f-8b2b-f82a84351fba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:54.441861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:54.442823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:54.443120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:54.443405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:54.457550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb6207a5aa043ee91638abe3a156a04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:54.461078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:54.463984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:54.482415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb6207a5aa043ee91638abe3a156a04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:54.485525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb6207a5aa043ee91638abe3a156a04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:54.521132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb6207a5aa043ee91638abe3a156a04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:54.526424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb6207a5aa043ee91638abe3a156a04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:54.535284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 867/1719] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:54.559670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 853/1720] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:54 2022] DELETE /identity/v3/users/8cb6207a5aa043ee91638abe3a156a04 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:54.714742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:54.716336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:54.740481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71198a-2e89-4471-b1f6-1820d3311f8a service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:54.992001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:55.063066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:55.067432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:55.067727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:55.088583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] PATH_INFO: `/v3/projects/42f1fb9950244410993a0e2101f4c56d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:55.091472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f1fb9950244410993a0e2101f4c56d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:55.094525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:55.234093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f1fb9950244410993a0e2101f4c56d', 'request_id': 'req-5354dc89-31e2-40c0-94f9-8fddcea545c4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.256654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f1fb9950244410993a0e2101f4c56d', 'request_id': 'req-5354dc89-31e2-40c0-94f9-8fddcea545c4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.284544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f1fb9950244410993a0e2101f4c56d', 'request_id': 'req-5354dc89-31e2-40c0-94f9-8fddcea545c4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.313999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] The token cache is being invalidate because project 42f1fb9950244410993a0e2101f4c56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:55.316539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.319656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5354dc89-31e2-40c0-94f9-8fddcea545c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.326695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 868/1721] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:54 2022] DELETE /identity/v3/projects/42f1fb9950244410993a0e2101f4c56d => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:55.332562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:55.335440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:55.373847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fbc14-b2af-4935-97d5-96a7f873fa02 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:55.487305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:55.487612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:55.488050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:55.533880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:55.535097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:55.536469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:55.536725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:55.537052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:55.537486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] PATH_INFO: `/v3/users/af5d230fb52b439cab8b0b3f20a3ac7f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:55.538723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5d230fb52b439cab8b0b3f20a3ac7f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:55.541810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:55.616940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:55.678570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5d230fb52b439cab8b0b3f20a3ac7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.726536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5d230fb52b439cab8b0b3f20a3ac7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.728923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5d230fb52b439cab8b0b3f20a3ac7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.747403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 869/1722] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:44:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:55.750243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5d230fb52b439cab8b0b3f20a3ac7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.752418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5d230fb52b439cab8b0b3f20a3ac7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:55.799586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 854/1723] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:55 2022] DELETE /identity/v3/users/af5d230fb52b439cab8b0b3f20a3ac7f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:55.939514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:55.941875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:55.970265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d406a0-c545-4e7a-81f8-fdadfc5da2b5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:56.028060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:56.034307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:56.100610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:56.101652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:56.102014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:56.102873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] PATH_INFO: `/v3/projects/ca2fc36fc2df4c2e927997ea5b196b5c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:56.106327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2fc36fc2df4c2e927997ea5b196b5c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:56.110544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:56.173900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:56.258560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2fc36fc2df4c2e927997ea5b196b5c', 'request_id': 'req-28c93489-80d0-47c2-9799-7c8f4afbd0ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.275196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2fc36fc2df4c2e927997ea5b196b5c', 'request_id': 'req-28c93489-80d0-47c2-9799-7c8f4afbd0ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.286912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2fc36fc2df4c2e927997ea5b196b5c', 'request_id': 'req-28c93489-80d0-47c2-9799-7c8f4afbd0ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.306848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac52046-4836-4565-a937-56405cc53cba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:56.313861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] The token cache is being invalidate because project ca2fc36fc2df4c2e927997ea5b196b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:56.314154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.315050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.319613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 870/1724] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:55 2022] DELETE /identity/v3/projects/ca2fc36fc2df4c2e927997ea5b196b5c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:56.320772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:56.327789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:56.476059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:56.478867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:56.480114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:56.480606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:56.489939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:56.535466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c93489-80d0-47c2-9799-7c8f4afbd0ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:56.567937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:56.568872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:56.569160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:56.569441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] PATH_INFO: `/v3/users/3030e96412994189b7c99532bdeda818` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:56.576282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3030e96412994189b7c99532bdeda818)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:56.580251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:56.643084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:56.688176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:56.705172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:56.727140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 855/1725] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:44:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:44:56.771312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3030e96412994189b7c99532bdeda818'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.771602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3030e96412994189b7c99532bdeda818'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.801736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3030e96412994189b7c99532bdeda818'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.803397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3030e96412994189b7c99532bdeda818'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.828813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5a9af7f3-5170-4a71-9328-85a2df641392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3030e96412994189b7c99532bdeda818'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:56.927448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 871/1726] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:56 2022] DELETE /identity/v3/users/3030e96412994189b7c99532bdeda818 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:58.344675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:58.346177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:58.378505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a3cd3-1b1e-4c2d-98c1-a40b66579cf2 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:58.386523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:58.387096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:58.387096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:58.443683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] MFA Rules not processed for user `6f44d10be1654d17990966ae64d43ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:58.449863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:58.526096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:58.584627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:58.588629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:58.588819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:58.589088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] PATH_INFO: `/v3/projects/ffee65c6a4d94585896ea903da1ab723` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:58.605162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffee65c6a4d94585896ea903da1ab723)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:58.607250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:58.614245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 872/1727] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 26 13:44:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:44:58.659821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:58.661420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:58.696362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1592-37f9-4a55-9022-6916a00eeb9b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:58.709742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:58.711038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:58.711326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:58.714615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] PATH_INFO: `/v3/projects/4d1e9baec0a64d06858f2f49ad3b6eab` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:58.716976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1e9baec0a64d06858f2f49ad3b6eab)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:58.724130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:58.736875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffee65c6a4d94585896ea903da1ab723', 'request_id': 'req-89d76d48-dae7-4221-b482-3f0429499cec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.753101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffee65c6a4d94585896ea903da1ab723', 'request_id': 'req-89d76d48-dae7-4221-b482-3f0429499cec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.779562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffee65c6a4d94585896ea903da1ab723', 'request_id': 'req-89d76d48-dae7-4221-b482-3f0429499cec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.800575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] The token cache is being invalidate because project ffee65c6a4d94585896ea903da1ab723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:58.800847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.802680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.815130 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 856/1728] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:58 2022] DELETE /identity/v3/projects/ffee65c6a4d94585896ea903da1ab723 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:58.816168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:58.819830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:58.870961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1e9baec0a64d06858f2f49ad3b6eab', 'request_id': 'req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.887414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1e9baec0a64d06858f2f49ad3b6eab', 'request_id': 'req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.906728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1e9baec0a64d06858f2f49ad3b6eab', 'request_id': 'req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.938227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] The token cache is being invalidate because project 4d1e9baec0a64d06858f2f49ad3b6eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:44:58.938523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.939602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-96c7ad0f-4ab7-40ca-beb1-58d365b4b1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:58.942092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 873/1729] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:44:58 2022] DELETE /identity/v3/projects/4d1e9baec0a64d06858f2f49ad3b6eab => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:58.944106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f9406-199c-434f-bf13-331a1e017890 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:58.944463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f9406-199c-434f-bf13-331a1e017890 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:58.944744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f9406-199c-434f-bf13-331a1e017890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:58.991412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0f6f9406-199c-434f-bf13-331a1e017890 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:58.992731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0f6f9406-199c-434f-bf13-331a1e017890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.019616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d76d48-dae7-4221-b482-3f0429499cec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:59.070104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:59.115965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:59.115965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.115965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.115965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.121435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.144044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:59.147707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8681b71-19dc-43cf-be07-3fc968ebb49f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:59.172637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 857/1730] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:44:59.174285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42300eea-0e35-49d2-a72f-37f0fcf2f2f5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.174718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42300eea-0e35-49d2-a72f-37f0fcf2f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.175125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42300eea-0e35-49d2-a72f-37f0fcf2f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.214223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-42300eea-0e35-49d2-a72f-37f0fcf2f2f5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:59.215607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-42300eea-0e35-49d2-a72f-37f0fcf2f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.245415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 874/1731] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:59.249333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.249805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.251260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.361751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] MFA Rules not processed for user `9958aaa8690c468da2acec47b2bb5f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:59.370322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.371048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 858/1732] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:59.373127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a903923-9a33-4971-978a-c291e90fff33 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.373899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a903923-9a33-4971-978a-c291e90fff33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.374395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a903923-9a33-4971-978a-c291e90fff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.422910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2a903923-9a33-4971-978a-c291e90fff33 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:59.424578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a903923-9a33-4971-978a-c291e90fff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.463826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 859/1733] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:59.466983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805bb42-2b73-4730-aaa6-6abbe987210e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.468486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805bb42-2b73-4730-aaa6-6abbe987210e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.469034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805bb42-2b73-4730-aaa6-6abbe987210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.531125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2805bb42-2b73-4730-aaa6-6abbe987210e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:59.533080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2805bb42-2b73-4730-aaa6-6abbe987210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.558240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 860/1734] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:59.560129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.560816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.561472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.579630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:59.624067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] MFA Rules not processed for user `dd8055e46e6149d3897fb88af0b19d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:59.625569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.626079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 875/1735] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:59.626939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:59.674956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4a654-4ccb-41eb-b478-b05c831a3149 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:59.700117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:59.700937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.701225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.701508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.702249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:59.702562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:59.708107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:59.726751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 861/1736] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:44:59.728885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:59.731156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.748571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0adfab-4c14-4e30-970a-6e68b7e93da0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:59.754810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:59.757122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.757601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.758284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] PATH_INFO: `/v3/users/7da0c375806b44c8ae6c8d622fc357d3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.759343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 876/1737] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:59.760335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:59.760767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da0c375806b44c8ae6c8d622fc357d3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:59.763593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:59.796670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490e0d3-8626-4ed9-beb9-82bd3fa027ab admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:44:59.802728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:44:59.804579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.806695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.808598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.810462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:44:59.813269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:44:59.818102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:44:59.840928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da0c375806b44c8ae6c8d622fc357d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:59.847867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:44:59.854986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 877/1738] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:44:59.856184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:44:59.860158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:44:59.864944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da0c375806b44c8ae6c8d622fc357d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:59.865942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da0c375806b44c8ae6c8d622fc357d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:59.880454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da0c375806b44c8ae6c8d622fc357d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:59.881878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ad4c8da-f63c-46b3-a46f-075516505b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da0c375806b44c8ae6c8d622fc357d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:44:59.922317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 862/1739] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:44:59 2022] DELETE /identity/v3/users/7da0c375806b44c8ae6c8d622fc357d3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:44:59.924613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:44:59.925823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:44:59.926537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:44:59.989271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] MFA Rules not processed for user `dd8055e46e6149d3897fb88af0b19d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:44:59.991238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:00.091082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d21181-048d-4ab7-9943-f29863ef928d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:00.179342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 863/1740] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:00.180199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:00.182225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:00.288584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc43cb-681b-4869-a4ac-683dd14fa65d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:00.296433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:00.297382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:00.297838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:00.298257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:00.299319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:00.301923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:00.366155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 864/1741] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:00 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:00.366439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:00.373202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:00.376397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:00.377128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:00.377506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:00.377903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:00.380054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:00.515253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39cc85-f43a-46fe-a28a-d95f81ff403e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:00.535152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:00.535375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:00.536082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:00.536396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:00.536718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:00.537285 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a85e-6076-4806-ab81-0aad57dffa7a service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:00.537627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:00.539624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:00.549747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:00.552472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:00.562986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 878/1742] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:44:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:00.564366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:00.564661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:00.568591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:00.638500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 865/1743] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:00 2022] POST /identity/v3/projects => generated 357 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:00.639590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:00.640507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:00.643610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:00.783364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 879/1744] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:00.796250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:00.840128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db98333-94cd-4ec1-bfb2-e14ceb1ceca5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:00.845084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851d62-f65b-4c37-8f71-386c17c201d5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:00.855435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:00.857675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:00.858749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:00.859049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:00.859805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:00.861857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:00.865938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:00.867602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:00.871673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:00.872765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:00.875782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:00.887409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:00.924267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 866/1745] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:00 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:00.929169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:00.933391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:00.964444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 880/1746] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:00 2022] POST /identity/v3/users => generated 279 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:00.964998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:00.979681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.010305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c661615-a4d9-4520-b240-8470d010fca3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.021589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.025642 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.026468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.027147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] PATH_INFO: `/v3/users/ccfe3bb99949474a86a74b4683f843be` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.029932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccfe3bb99949474a86a74b4683f843be)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.036016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.157191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccfe3bb99949474a86a74b4683f843be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:01.160476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccfe3bb99949474a86a74b4683f843be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:01.197913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccfe3bb99949474a86a74b4683f843be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:01.200655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea82464-63c7-4ffc-abc7-b96bc75bb106 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.202100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccfe3bb99949474a86a74b4683f843be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:01.217574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.218268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.218555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.218836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.225922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.240274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccfe3bb99949474a86a74b4683f843be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:01.344658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 881/1747] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:00 2022] DELETE /identity/v3/users/ccfe3bb99949474a86a74b4683f843be => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:01.346047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:01.351834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.465861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.470954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.512744 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 867/1748] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:01.517847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:01.523700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.573526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb9ca5-3e55-4134-90b3-d8c9bbe8f5f0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.591400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40f708-84f1-4633-9af0-bab2d8b6a0f8 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.595402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.599891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.602341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.603492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.613779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.636467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:01.663777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:01.769902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.771813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.785053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 882/1749] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:01.786405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:01.788105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.802800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06712-ffc5-4818-94e0-fdffc5c33af9 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.811090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e038e-f818-4bcf-a82e-815a462509d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.812040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e038e-f818-4bcf-a82e-815a462509d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.812591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e038e-f818-4bcf-a82e-815a462509d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.813078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e038e-f818-4bcf-a82e-815a462509d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.814138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e038e-f818-4bcf-a82e-815a462509d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.815994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e038e-f818-4bcf-a82e-815a462509d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.829086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 883/1750] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:01.830967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.831467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.831951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.854186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.854881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.855122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:01.855232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.855434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.856172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.857086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:01.857860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.871367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 884/1751] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:01.874895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:01.890393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d209e0-407c-4a8c-a73d-a6001cd76672 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.894843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 868/1752] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:01.897634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.898432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.898783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.899847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.900953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:01.901774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.903414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.919048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 885/1753] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:01.920695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:01.938323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c33be-4a3a-4d34-81f8-d43ae77b1684 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.943781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6f1d-cec8-453f-acfe-9d71af282576 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:01.951999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.952718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.953006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.953287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.954020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:01.957339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:01.958120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdd096-32e3-456f-9198-13dac54ddd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:01.959204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdd096-32e3-456f-9198-13dac54ddd7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:01.959476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdd096-32e3-456f-9198-13dac54ddd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:01.959748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdd096-32e3-456f-9198-13dac54ddd7b admin admin] PATH_INFO: `/v3/projects/02f7c8a7b320491eae95111f980252ac/users/edcad966cb144ae0a711e2589058a873/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:01.992536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 869/1754] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:01.995200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:02.000598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdd096-32e3-456f-9198-13dac54ddd7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f7c8a7b320491eae95111f980252ac, user_id=edcad966cb144ae0a711e2589058a873, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:02.005423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdd096-32e3-456f-9198-13dac54ddd7b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:02.014995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bd4c6-67dc-4663-b8c2-9031d47911cb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:02.023609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7eb53-16fd-48a5-84eb-0b3aed00cb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:02.023923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7eb53-16fd-48a5-84eb-0b3aed00cb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:02.024419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7eb53-16fd-48a5-84eb-0b3aed00cb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:02.024907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7eb53-16fd-48a5-84eb-0b3aed00cb2e admin admin] PATH_INFO: `/v3/projects/e5d624ff4f9b4b0ab41435153eec5f1e/users/0982719a4a96494dbcc671eb419c5b93/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:02.044367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 886/1755] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:01 2022] PUT /identity/v3/projects/02f7c8a7b320491eae95111f980252ac/users/edcad966cb144ae0a711e2589058a873/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:02.048821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:02.049326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:02.049761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:02.095740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7eb53-16fd-48a5-84eb-0b3aed00cb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d624ff4f9b4b0ab41435153eec5f1e, user_id=0982719a4a96494dbcc671eb419c5b93, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:02.107600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7eb53-16fd-48a5-84eb-0b3aed00cb2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:02.150215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] MFA Rules not processed for user `edcad966cb144ae0a711e2589058a873`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:02.151567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:02.226737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 870/1756] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:01 2022] PUT /identity/v3/projects/e5d624ff4f9b4b0ab41435153eec5f1e/users/0982719a4a96494dbcc671eb419c5b93/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:02.241225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:02.241596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:02.244398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:02.342387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 887/1757] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:02 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:02.342896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:02.346544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:02.362437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] MFA Rules not processed for user `0982719a4a96494dbcc671eb419c5b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:02.363783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:02.468468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:02.502154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ccb85-523d-44ad-88cf-42e81d86c2b0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:02.518552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:02.519238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:02.532447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:02.532694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:02.594440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 871/1758] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:02.595459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:02.766762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:02.768259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:02.817096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 888/1759] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:02.817989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:02.842870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512683a-9fdc-473c-8360-08aae638e759 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:02.862568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:02.862843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:02.863138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:02.863424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:02.864196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:02.866205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:02.873804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96a13a-ebc9-4ac6-b464-83b686433c29 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:02.894570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:02.896372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:02.896866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:02.898232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:02.899758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:02.902076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:02.922996 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:02.945562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 889/1760] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:02 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:02.947506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:02.986139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 872/1761] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:02 2022] POST /identity/v3/projects => generated 357 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:02.988371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.156253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f23768-5bdc-44e3-b83b-928e517c2dad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.174364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.174950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.175246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.175521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.176222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.178199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.193143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c6d-33e8-4981-8040-6b9406096a6e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.221792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 890/1762] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:02 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:03.228018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.228456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.230246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.231725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.232365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.233456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.236026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.265863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3433da-2788-4009-9b21-b4d208ff0202 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.282892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 873/1763] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:02 2022] POST /identity/v3/users => generated 285 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:03.294138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.297411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.298121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.298393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.298795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.299392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.301064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.314038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19cd2a-a997-4e38-886c-6e0a3957d5bf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.348112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 891/1764] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:03.349271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.358704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.359426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.359758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.360058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.361568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.371921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.386698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcf85-e961-4634-8c02-f5147945adee admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.400554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 874/1765] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:03.408811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.415577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.416504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.416857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.420749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] PATH_INFO: `/v3/projects/fbc5059a12f24221b2df7c88b9a75a5a/users/a5c5e151b06b41d29058bb904e8e3595/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.429955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb875005-81df-4beb-97e7-db3aa26114f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.448028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.453567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.454920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.456330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] PATH_INFO: `/v3/projects/484ddde9dfe34888b27ce2cd1a550f49/users/2509f223b9a74ee48166d6cf2e170b08/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.470715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc5059a12f24221b2df7c88b9a75a5a, user_id=a5c5e151b06b41d29058bb904e8e3595, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.476840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.515442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 892/1766] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:03 2022] PUT /identity/v3/projects/fbc5059a12f24221b2df7c88b9a75a5a/users/a5c5e151b06b41d29058bb904e8e3595/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:03.527720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.651934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:03.792749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85c142-c2b6-48df-b5d9-8b3707fe20ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.806537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.807024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484ddde9dfe34888b27ce2cd1a550f49, user_id=2509f223b9a74ee48166d6cf2e170b08, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.809043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.809785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.823483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.823832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.824767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.826969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:03.841002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 875/1767] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:03 2022] PUT /identity/v3/projects/484ddde9dfe34888b27ce2cd1a550f49/users/2509f223b9a74ee48166d6cf2e170b08/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:03.842149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.854944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 893/1768] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:03.856194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:03.884855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b55f7-6bab-495c-8d6b-40d93d4e6fbb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.950333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d926e-e59d-49cc-8d4d-90e782817468 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:03.963867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:03.965032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:03.966328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:03.967216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:03.970875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:03.973779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:04.020532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 876/1769] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:04.021432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:04.072232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61659737-d5de-4a84-95d2-fa542259a6a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:04.095875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecafe65-7c8b-4604-8632-69b8d1e85dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:04.098531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecafe65-7c8b-4604-8632-69b8d1e85dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:04.098681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecafe65-7c8b-4604-8632-69b8d1e85dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:04.098954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecafe65-7c8b-4604-8632-69b8d1e85dc3 admin admin] PATH_INFO: `/v3/projects/484ddde9dfe34888b27ce2cd1a550f49/users/2509f223b9a74ee48166d6cf2e170b08/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:04.111118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecafe65-7c8b-4604-8632-69b8d1e85dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484ddde9dfe34888b27ce2cd1a550f49, user_id=2509f223b9a74ee48166d6cf2e170b08, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:04.120916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecafe65-7c8b-4604-8632-69b8d1e85dc3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:04.171124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4585-968f-4059-a681-7bd7ac43d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:04.172173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4585-968f-4059-a681-7bd7ac43d6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:04.173097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 877/1770] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:04 2022] PUT /identity/v3/projects/484ddde9dfe34888b27ce2cd1a550f49/users/2509f223b9a74ee48166d6cf2e170b08/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:04.173369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4585-968f-4059-a681-7bd7ac43d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:04.173675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4585-968f-4059-a681-7bd7ac43d6fc admin admin] PATH_INFO: `/v3/projects/fbc5059a12f24221b2df7c88b9a75a5a/users/a5c5e151b06b41d29058bb904e8e3595/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:04.175832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4585-968f-4059-a681-7bd7ac43d6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc5059a12f24221b2df7c88b9a75a5a, user_id=a5c5e151b06b41d29058bb904e8e3595, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:04.178864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4585-968f-4059-a681-7bd7ac43d6fc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:04.186575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:04.187042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:04.187215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:04.247492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 894/1771] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:03 2022] PUT /identity/v3/projects/fbc5059a12f24221b2df7c88b9a75a5a/users/a5c5e151b06b41d29058bb904e8e3595/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:04.249708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:04.250013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:04.250297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:04.293940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] MFA Rules not processed for user `2509f223b9a74ee48166d6cf2e170b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:04.298682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:04.338288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:04.345209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] MFA Rules not processed for user `a5c5e151b06b41d29058bb904e8e3595`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:04.346689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:04.516707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 878/1772] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:04 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:04.517755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:04.519214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:04.646103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 895/1773] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:04.661175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:04.666627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:04.784869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b0b90-1f9c-4cb0-b7e6-f962cf2d5567 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:04.804243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:04.806056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:04.806235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:04.806504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] PATH_INFO: `/v3/projects/89ba51f0146e47ad9405899306d304db` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:04.813020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ba51f0146e47ad9405899306d304db)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:04.822860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:04.830660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:04.851451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20d5b0-42a2-45b8-90c8-6494897c388f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:04.856951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:04.857866 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:04.858385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:04.859179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:04.875509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:04.892596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:04.929905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 896/1774] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:04 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:04.931520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:04.939561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:04.988692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16519-691e-4772-99ce-4da9465b2eaf service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:05.066648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ba51f0146e47ad9405899306d304db', 'request_id': 'req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.124222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ba51f0146e47ad9405899306d304db', 'request_id': 'req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.145728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ba51f0146e47ad9405899306d304db', 'request_id': 'req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.165909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] The token cache is being invalidate because project 89ba51f0146e47ad9405899306d304db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:05.166205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.169452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.174796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 879/1775] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:04 2022] DELETE /identity/v3/projects/89ba51f0146e47ad9405899306d304db => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:05.182615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:05.192916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:05.249542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:05.256233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:05.260881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:05.261219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:05.272588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:05.443413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:05.538071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96a46f-f0f5-4e1b-abf5-de44be0307e6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:05.552065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:05.552911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:05.553190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:05.553467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] PATH_INFO: `/v3/users/890a6f879e3e492dabf3d4a2aa45fcd6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:05.557322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=890a6f879e3e492dabf3d4a2aa45fcd6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:05.559272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:05.566115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:05.568587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:05.598843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 897/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:05.601544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:05.608791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:05.652988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7fb18-2ef4-4537-a1a6-370a1890a0f1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:05.661388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:05.662980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:05.663285 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:05.663651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:05.670089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:05.771421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890a6f879e3e492dabf3d4a2aa45fcd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.834462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890a6f879e3e492dabf3d4a2aa45fcd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.839397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890a6f879e3e492dabf3d4a2aa45fcd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.863313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890a6f879e3e492dabf3d4a2aa45fcd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.867038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e47d8f1-132b-4103-81f7-c0b23dff99f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890a6f879e3e492dabf3d4a2aa45fcd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:05.958357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 880/1777] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:05 2022] DELETE /identity/v3/users/890a6f879e3e492dabf3d4a2aa45fcd6 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:05.959949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e7dc6-6558-4845-b7dd-2229d0cb750f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:05.960240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e7dc6-6558-4845-b7dd-2229d0cb750f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:05.960635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e7dc6-6558-4845-b7dd-2229d0cb750f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:05.962305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 881/1778] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:05.981279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:05.981572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:05.982303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:05.983843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f5667-cefa-4186-af85-cc9f39a4d15c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:05.985518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:06.015061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 898/1779] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:06.050054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:06.050388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:06.050699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:06.069586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] MFA Rules not processed for user `dd8055e46e6149d3897fb88af0b19d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:06.089927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:06.101072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:06.103236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:06.305789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:06.320126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 882/1780] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:06.485235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 899/1781] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:45:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:06.921891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:06.923666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:07.238434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e888d4-e9eb-4d40-bdee-1d359f7c2ee7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:07.249287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:07.252934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:07.253250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:07.253550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] PATH_INFO: `/v3/projects/a124b343b9cb4e7b85a6c75d83c01589` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:07.277312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a124b343b9cb4e7b85a6c75d83c01589)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:07.279342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:07.315251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:07.437506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a124b343b9cb4e7b85a6c75d83c01589', 'request_id': 'req-525461b2-3b08-4ede-8218-8bdc225d6ca9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.445538 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a124b343b9cb4e7b85a6c75d83c01589', 'request_id': 'req-525461b2-3b08-4ede-8218-8bdc225d6ca9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.460062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a124b343b9cb4e7b85a6c75d83c01589', 'request_id': 'req-525461b2-3b08-4ede-8218-8bdc225d6ca9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.494065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] The token cache is being invalidate because project a124b343b9cb4e7b85a6c75d83c01589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:07.494369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.496838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.500280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 883/1782] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:06 2022] DELETE /identity/v3/projects/a124b343b9cb4e7b85a6c75d83c01589 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:07.504697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:07.535144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:07.537129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:07.542392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667abf5d-a280-4176-8e77-5ba300e1a8bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:07.772998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:07.774027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:07.774313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:07.774597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] PATH_INFO: `/v3/users/1688ea1a38bb40ee91935065edcd3bb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:07.787519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1688ea1a38bb40ee91935065edcd3bb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:07.790429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:07.845870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525461b2-3b08-4ede-8218-8bdc225d6ca9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:07.864425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:07.865210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:07.865513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:07.865857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:07.870228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:07.908218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:07.913661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df3b1c-2d19-44c0-9e3c-47dd203b0c46 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:07.918227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1688ea1a38bb40ee91935065edcd3bb4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.918738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1688ea1a38bb40ee91935065edcd3bb4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.934455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1688ea1a38bb40ee91935065edcd3bb4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.936279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1688ea1a38bb40ee91935065edcd3bb4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:07.943585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 884/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:07.953272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f49cd85c-c592-42f0-918a-abb66b414aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1688ea1a38bb40ee91935065edcd3bb4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:08.050422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 900/1784] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:07 2022] DELETE /identity/v3/users/1688ea1a38bb40ee91935065edcd3bb4 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:08.216780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbac31-b9e7-4401-8615-b5d166ed16a0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:08.217180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbac31-b9e7-4401-8615-b5d166ed16a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:08.217468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbac31-b9e7-4401-8615-b5d166ed16a0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:08.218997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 885/1785] 10.222.0.16 () {54 vars in 912 bytes} [Thu May 26 13:45:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:08.245425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:08.246322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:08.246745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:08.312716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] MFA Rules not processed for user `6f44d10be1654d17990966ae64d43ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:08.314022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:08.481071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:08.510386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 901/1786] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:45:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:08.654207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbac31-b9e7-4401-8615-b5d166ed16a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:08.658215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-00dbac31-b9e7-4401-8615-b5d166ed16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:08.701448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbac31-b9e7-4401-8615-b5d166ed16a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:08.710690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701f7d8-a9db-46fc-be44-398247f840c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:08.712515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701f7d8-a9db-46fc-be44-398247f840c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:08.714717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701f7d8-a9db-46fc-be44-398247f840c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:08.718104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701f7d8-a9db-46fc-be44-398247f840c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:08.741524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701f7d8-a9db-46fc-be44-398247f840c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:08.749241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701f7d8-a9db-46fc-be44-398247f840c0 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:08.769027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 886/1787] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:08.833455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:08.835535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:09.093516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46378a58-5e49-4e30-bceb-db8d267794f5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:09.114586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:09.124678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:09.125430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:09.125632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] PATH_INFO: `/v3/projects/b055bade54d04efebc2155a071837b7b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:09.128735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b055bade54d04efebc2155a071837b7b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:09.131601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:09.234465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b055bade54d04efebc2155a071837b7b', 'request_id': 'req-057617e8-04d6-4ffd-a600-6cbba7d9ed00'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:09.254522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b055bade54d04efebc2155a071837b7b', 'request_id': 'req-057617e8-04d6-4ffd-a600-6cbba7d9ed00'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:09.270747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b055bade54d04efebc2155a071837b7b', 'request_id': 'req-057617e8-04d6-4ffd-a600-6cbba7d9ed00'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:09.306599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] The token cache is being invalidate because project b055bade54d04efebc2155a071837b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:09.311357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:09.312908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-057617e8-04d6-4ffd-a600-6cbba7d9ed00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:09.319111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 902/1788] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:08 2022] DELETE /identity/v3/projects/b055bade54d04efebc2155a071837b7b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:09.325836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:09.327384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:09.327928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:09.402475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:09.403730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:09.544316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 887/1789] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:09.566846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:09.567719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:09.568643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:09.675698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:09.681224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:09.738156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 903/1790] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:09.742452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:09.759904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c89da-a8d8-4fbc-be96-b788c3ce19ea None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:09.773063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:09.773803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:09.774098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:09.774578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:09.775326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:09.775636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:09.784208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:09.789872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:09.825952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:09.835658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b9b7e-1e72-4af7-931c-e51170a1b5ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:09.836575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 888/1791] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:09.848708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:09.849468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:09.849863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:09.850025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:09.850721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:09.855590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:09.890939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 904/1792] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:09 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:09.895428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:09.908135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f822247-c056-42ae-bb90-5286143567fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.017004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:10.017976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.018425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.018850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.019920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:10.022530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:10.083531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:10.084045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 889/1793] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:09 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:10.097018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2f404-a50c-4311-a9fc-e7d4d624f54c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.105665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:10.107402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.107702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.107980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.109131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:10.111988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:10.142322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:10.155260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 905/1794] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:10.161894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b142-51cf-43da-a62c-37af9fe5ca2d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.168809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778baf1-0f2b-4a3f-a917-9a8faa491385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:10.169848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778baf1-0f2b-4a3f-a917-9a8faa491385 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.170140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778baf1-0f2b-4a3f-a917-9a8faa491385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.170417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778baf1-0f2b-4a3f-a917-9a8faa491385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.171965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778baf1-0f2b-4a3f-a917-9a8faa491385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:10.174732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778baf1-0f2b-4a3f-a917-9a8faa491385 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:10.199358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 890/1795] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:10.199554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:10.224075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587c6ba-16d2-4109-8302-b6b2c07607ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.232246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:10.260793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.261247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.261847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] PATH_INFO: `/v3/projects/0484da23a9ab4d22aa654f0874f1b632/users/4433005759624d9ab32457f753d07b11/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.295468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0484da23a9ab4d22aa654f0874f1b632, user_id=4433005759624d9ab32457f753d07b11, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:10.311493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:10.348444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 906/1796] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:10 2022] PUT /identity/v3/projects/0484da23a9ab4d22aa654f0874f1b632/users/4433005759624d9ab32457f753d07b11/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:10.358854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.360515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.360515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.418605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] MFA Rules not processed for user `4433005759624d9ab32457f753d07b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:10.420205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:10.471961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:10.474103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:10.549391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:10.626202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 891/1797] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:10.633768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:10.670577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cab94-482e-4427-a5d5-e4042b4f6687 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.676028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:10.677698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.678200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.678819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.679673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:10.683023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:10.692234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387311dd-5832-4129-b4fb-33ae7416ddf9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.709170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:10.710210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.710693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.711266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] PATH_INFO: `/v3/projects/4b6885417b824a37a7beec06c9942bb0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:10.712828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6885417b824a37a7beec06c9942bb0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:10.715495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:10.729141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 892/1798] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:10 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:10.730338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:10.837625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6885417b824a37a7beec06c9942bb0', 'request_id': 'req-c98a148e-6497-4555-95ff-97eb3e18e1d1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:10.851762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:10.863951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6885417b824a37a7beec06c9942bb0', 'request_id': 'req-c98a148e-6497-4555-95ff-97eb3e18e1d1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:10.892485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa03d2-29d8-4fc6-a898-d8eb9cbaf04b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:10.895244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6885417b824a37a7beec06c9942bb0', 'request_id': 'req-c98a148e-6497-4555-95ff-97eb3e18e1d1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:10.963187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] The token cache is being invalidate because project 4b6885417b824a37a7beec06c9942bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:10.963512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:10.964128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c98a148e-6497-4555-95ff-97eb3e18e1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:10.965740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 907/1799] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:10 2022] DELETE /identity/v3/projects/4b6885417b824a37a7beec06c9942bb0 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:10.972123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e87fd-651d-4a22-865c-bcd69c55d305 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:10.972518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e87fd-651d-4a22-865c-bcd69c55d305 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:10.973070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e87fd-651d-4a22-865c-bcd69c55d305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.089821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c05e87fd-651d-4a22-865c-bcd69c55d305 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:11.091079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.091699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.091972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.092473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.093287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.095940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.111492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c05e87fd-651d-4a22-865c-bcd69c55d305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:11.139719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 908/1800] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:11.152173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.152671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.153239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.169231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 893/1801] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:10 2022] POST /identity/v3/users => generated 294 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:11.172716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.179911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:11.205085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:11.205532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab49010-4e16-4717-ae88-f291e22ea544 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.208467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:11.224030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.225655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.226218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.226824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.228866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.231811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.234845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 909/1802] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:11.238695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.268021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ed3c-b4d9-42d5-885e-22bb894e8f44 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.282490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.284139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 894/1803] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:11.287244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.287514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.287631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.288088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.295765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:11.297114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.301429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.319015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af398ddb-51a6-42f9-b3c7-c74b8efda100 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.347490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 910/1804] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:11.348528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.365165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.368955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.369959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.375329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3d64-ce41-46a8-b4bd-3ee09add41f4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.379370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] PATH_INFO: `/v3/projects/e536c0e1e08340e89b27fd10934f4843/users/01a6117f9d5948a1940547079b1904f6/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.392382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.393225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.393521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.393812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.394562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.402225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.417676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e536c0e1e08340e89b27fd10934f4843, user_id=01a6117f9d5948a1940547079b1904f6, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.423391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.457107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 895/1805] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:11 2022] PUT /identity/v3/projects/e536c0e1e08340e89b27fd10934f4843/users/01a6117f9d5948a1940547079b1904f6/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:11.466192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.474782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 911/1806] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:11 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:11.475687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.609086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4381-8e11-4bbe-9ebf-5f542602e068 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.615297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.616850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.618521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.619174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.620108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.622631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3f80-ade3-442b-b397-5147d41edf28 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.623254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.631926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.632734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.633064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.633747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.634991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.637327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.639146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:11.662659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 912/1807] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:11 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:11.664778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.665782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 896/1808] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:11.670508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.684737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85add834-b39f-4800-a2d2-07272f3aacfb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.685723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef34412-4da1-4e26-bef6-38a0f3f1a75e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.692455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.693861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.694444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.694745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.695993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.696455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50100d54-f435-491e-b479-9f64fed64198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.698570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.699321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50100d54-f435-491e-b479-9f64fed64198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.700424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50100d54-f435-491e-b479-9f64fed64198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.700988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50100d54-f435-491e-b479-9f64fed64198 admin admin] PATH_INFO: `/v3/projects/e536c0e1e08340e89b27fd10934f4843/users/01a6117f9d5948a1940547079b1904f6/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.706771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50100d54-f435-491e-b479-9f64fed64198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e536c0e1e08340e89b27fd10934f4843, user_id=01a6117f9d5948a1940547079b1904f6, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.713399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50100d54-f435-491e-b479-9f64fed64198 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.730279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 913/1809] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:11.731911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.749305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a869025-e56d-49dc-94e6-89953aeab7a2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.772682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 897/1810] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:11 2022] PUT /identity/v3/projects/e536c0e1e08340e89b27fd10934f4843/users/01a6117f9d5948a1940547079b1904f6/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:11.776382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.777263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.777955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.824765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] MFA Rules not processed for user `01a6117f9d5948a1940547079b1904f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:11.826578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:11.843861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:11.913873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.913873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.913873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.913873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:11.915965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:11.917768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:11.936962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 914/1811] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:11.946398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.969436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 898/1812] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:11.978447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a637218-c71e-436e-bf98-66e017d4b7bd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:11.984347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:11.987086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:11.990237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5b5ea-b39e-4cd4-b872-0f43c9c136b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:11.991180 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5b5ea-b39e-4cd4-b872-0f43c9c136b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:11.991466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5b5ea-b39e-4cd4-b872-0f43c9c136b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:11.991746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5b5ea-b39e-4cd4-b872-0f43c9c136b2 admin admin] PATH_INFO: `/v3/projects/f0eb112ea8a944ae8786052628ee74a9/users/f641458677c5400eb5ba78b9f866bed8/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.021356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5b5ea-b39e-4cd4-b872-0f43c9c136b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0eb112ea8a944ae8786052628ee74a9, user_id=f641458677c5400eb5ba78b9f866bed8, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.030117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5b5ea-b39e-4cd4-b872-0f43c9c136b2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.056361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 915/1813] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:11 2022] PUT /identity/v3/projects/f0eb112ea8a944ae8786052628ee74a9/users/f641458677c5400eb5ba78b9f866bed8/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:12.065444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8cc01-4108-42fc-93ad-5565a8e6fe97 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.065818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8cc01-4108-42fc-93ad-5565a8e6fe97 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.066187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8cc01-4108-42fc-93ad-5565a8e6fe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.072817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68390089-96d5-4fc9-ad97-6869026ae976 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.097297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d7f8cc01-4108-42fc-93ad-5565a8e6fe97 None None] MFA Rules not processed for user `f641458677c5400eb5ba78b9f866bed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:12.098731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d7f8cc01-4108-42fc-93ad-5565a8e6fe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:12.151403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.152087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.152534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.152810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.154866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:12.191192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 916/1814] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:12.191192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.191192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.191192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.213416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] MFA Rules not processed for user `9958aaa8690c468da2acec47b2bb5f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:12.214301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:12.246694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.250653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.267609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:12.271694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 899/1815] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:12.273088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:12.308582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 917/1816] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:12.309341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:12.363542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f8005-e7eb-459f-ba8a-4b4c81bcbe3e service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.377709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.378690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.379154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.380178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.382544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.387968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.458558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 900/1817] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:12 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:12.462886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:12.482991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83474da-dd27-416f-8035-ff25b97a073b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.581760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.583542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.583542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.583542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.583838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.585692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.627348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 918/1818] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:12 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:12.631551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:12.650222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd5d57-cc36-449f-8596-24a2a7c8b6ff admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.659080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.660064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.660500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.660831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.661638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.665821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.711967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:12.714129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 901/1819] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:12.715163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:12.715955 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:12.734675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339de1e-1fd6-450e-a351-05ede5aad24e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.741793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.743919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.744216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.744583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.745332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.748338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.767033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 902/1820] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:12.768537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:12.786026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdfb00-59ce-41c7-90f0-0918ca935a54 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.792740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff08d5-49c9-4865-8c01-6496e71e70b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.793647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff08d5-49c9-4865-8c01-6496e71e70b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.793944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff08d5-49c9-4865-8c01-6496e71e70b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.794229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff08d5-49c9-4865-8c01-6496e71e70b3 admin admin] PATH_INFO: `/v3/projects/5ce6bf92cca04078814abd290002f098/users/6bc8261177cb4fa5b5e8157f5ef3ec45/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.819263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff08d5-49c9-4865-8c01-6496e71e70b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce6bf92cca04078814abd290002f098, user_id=6bc8261177cb4fa5b5e8157f5ef3ec45, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.825586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff08d5-49c9-4865-8c01-6496e71e70b3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.830261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f09b-d3db-440e-a5d0-b43a41dc1d2a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:12.840893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:12.842713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.843384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.843872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.844615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 903/1821] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:12 2022] PUT /identity/v3/projects/5ce6bf92cca04078814abd290002f098/users/6bc8261177cb4fa5b5e8157f5ef3ec45/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:12.846602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:12.856701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:12.857602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:12.858294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:12.893760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] MFA Rules not processed for user `6bc8261177cb4fa5b5e8157f5ef3ec45`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:12.894992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:12.962008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:12.963538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:12.976697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 919/1822] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:45:12.982998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 904/1823] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:12.988726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.010220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5de2-8ae6-4420-bf4c-542725ab49c3 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.041595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:13.096176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.097436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.098133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.098571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.099440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.104359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:13.147446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 920/1824] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:12 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:13.148252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.166513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3dc0b-52a9-4444-a5b4-e40e0801422e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.279461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:13.306771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.307419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.307709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.308576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.309467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.312716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:13.356244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 905/1825] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:13 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:13.360472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.382281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb3e12-20e9-4a09-9b54-1fbf38cb3162 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.390129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.390879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.391175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.391521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.392262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.394106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:13.411094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 921/1826] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:13.416040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.442378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857a9b5-a9a5-4de5-a8e4-ab36c5f9aba9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.447875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.449115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.449401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.449885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] PATH_INFO: `/v3/projects/baf8082dddc2410b8242f99b7fe6f27b/users/4460e5f9d413402da059a6d8d84c4aba/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.468251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf8082dddc2410b8242f99b7fe6f27b, user_id=4460e5f9d413402da059a6d8d84c4aba, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.472100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:13.500626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 906/1827] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:13 2022] PUT /identity/v3/projects/baf8082dddc2410b8242f99b7fe6f27b/users/4460e5f9d413402da059a6d8d84c4aba/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:13.504349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.526744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c5f1-4b29-484d-9f82-a6de8466abca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.638786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ca8b0-d0a8-4bf1-bcd6-5f312e6bf022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.639788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca8b0-d0a8-4bf1-bcd6-5f312e6bf022 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.640126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca8b0-d0a8-4bf1-bcd6-5f312e6bf022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.640523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca8b0-d0a8-4bf1-bcd6-5f312e6bf022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.641379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ca8b0-d0a8-4bf1-bcd6-5f312e6bf022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.643058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ca8b0-d0a8-4bf1-bcd6-5f312e6bf022 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:13.654723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 922/1828] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:13.658983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.672236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a985-5732-4bce-bc22-70d4815a8f09 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.677447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.678580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.678895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.679378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] PATH_INFO: `/v3/projects/baf8082dddc2410b8242f99b7fe6f27b/users/4460e5f9d413402da059a6d8d84c4aba/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.682324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf8082dddc2410b8242f99b7fe6f27b, user_id=4460e5f9d413402da059a6d8d84c4aba, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.685095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:13.698228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 907/1829] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:13 2022] PUT /identity/v3/projects/baf8082dddc2410b8242f99b7fe6f27b/users/4460e5f9d413402da059a6d8d84c4aba/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:13.709241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3e434-a8dd-40d8-8aba-266bdc7dd5a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.709538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3e434-a8dd-40d8-8aba-266bdc7dd5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.709821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3e434-a8dd-40d8-8aba-266bdc7dd5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.740983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-aac3e434-a8dd-40d8-8aba-266bdc7dd5a2 None None] MFA Rules not processed for user `4460e5f9d413402da059a6d8d84c4aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:13.742844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aac3e434-a8dd-40d8-8aba-266bdc7dd5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:13.849408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 923/1830] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:13.864905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:13.951135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ff85-f971-4565-8da7-b9856565239e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:13.956625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bf96c-6ab9-41b1-b857-ef56c5c58ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:13.957399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf96c-6ab9-41b1-b857-ef56c5c58ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:13.957754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf96c-6ab9-41b1-b857-ef56c5c58ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:13.958170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf96c-6ab9-41b1-b857-ef56c5c58ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:13.972914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bf96c-6ab9-41b1-b857-ef56c5c58ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:13.975224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bf96c-6ab9-41b1-b857-ef56c5c58ba1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:14.001376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 908/1831] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:14.942401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fd3aa-1f89-4fba-9299-fef3d284a2f0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:14.943422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fd3aa-1f89-4fba-9299-fef3d284a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:14.944251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fd3aa-1f89-4fba-9299-fef3d284a2f0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:14.948527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 924/1832] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:45:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:14.962327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:14.962794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:14.963086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:14.990189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:14.992877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:15.094370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 909/1833] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:45:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:15.110054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fd3aa-1f89-4fba-9299-fef3d284a2f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:15.112104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-774fd3aa-1f89-4fba-9299-fef3d284a2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:15.138298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fd3aa-1f89-4fba-9299-fef3d284a2f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:15.144967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:15.146403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:15.146991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:15.147416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:15.169863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:15.172413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:15.181865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:15.186546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 925/1834] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:15.203212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81670d54-1f18-4743-876a-9b281e5ce360 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:15.213954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:15.214617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:15.214907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:15.215191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:15.259764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:15.297977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:15.300376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:15.315521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 910/1835] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:19.540191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:19.549550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774c48-5b77-409e-a0f5-3ac0fe69ccfa service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:19.553528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:19.554180 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:19.555261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:19.555599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:19.565559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:19.628596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:19.630125 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:19.640615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 926/1836] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:25.310068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:26.911236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:26.912019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:26.915064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:26.931271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f789ec-da09-4e37-bdfb-fa749cef32f1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:26.937790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7035f-9df5-459b-9b96-8e5148e2bc83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:26.938586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7035f-9df5-459b-9b96-8e5148e2bc83 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:26.938912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7035f-9df5-459b-9b96-8e5148e2bc83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:26.939259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7035f-9df5-459b-9b96-8e5148e2bc83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.027930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7035f-9df5-459b-9b96-8e5148e2bc83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:27.029553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7035f-9df5-459b-9b96-8e5148e2bc83 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:27.041524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8013e-fa3e-4ad9-9a6d-2e1c29cff9d5 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:27.047744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:27.049131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.049727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.050789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] PATH_INFO: `/v3/users/0982719a4a96494dbcc671eb419c5b93` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.052767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 911/1837] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:27.053046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0982719a4a96494dbcc671eb419c5b93)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:27.055495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a2e98-f81e-4f10-9a48-514cd4d7a4a7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.055495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a2e98-f81e-4f10-9a48-514cd4d7a4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.055495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a2e98-f81e-4f10-9a48-514cd4d7a4a7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.057432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:27.062131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 912/1838] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:45:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:45:27.076643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.076886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.077276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.106329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:27.111368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:27.135144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 913/1839] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:27.140752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0982719a4a96494dbcc671eb419c5b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.144230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0982719a4a96494dbcc671eb419c5b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.153882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:27.164502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0982719a4a96494dbcc671eb419c5b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.165226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f249294-2c8c-42b7-8fb5-912f8981acef None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:27.167369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0982719a4a96494dbcc671eb419c5b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.171486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:27.172453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.172924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.173385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.175535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:27.182339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0982719a4a96494dbcc671eb419c5b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.190229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:27.191913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283be948-f9a7-4969-9a4d-cbc038a6ad77 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:27.204968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 914/1840] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:45:27.215119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 927/1841] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:26 2022] DELETE /identity/v3/users/0982719a4a96494dbcc671eb419c5b93 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:27.217590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.224926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.227097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.256464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:27.257897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:27.330272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 915/1842] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:27.341609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:27.343284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:27.386870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:27.427781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69664eec-a93d-4352-8d45-4e7b957f9012 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:27.431870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:27.432565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.432849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.433130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.443496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:27.445230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:27.456233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 928/1843] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:27.732972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:27.734471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:27.748753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20a660-85d8-4119-9935-84bb84d14385 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:27.756578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:27.757653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.757968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.758413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] PATH_INFO: `/v3/projects/e5d624ff4f9b4b0ab41435153eec5f1e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.759486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d624ff4f9b4b0ab41435153eec5f1e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:27.761662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:27.815343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d624ff4f9b4b0ab41435153eec5f1e', 'request_id': 'req-c8e91cac-3d13-4d4c-8665-91f1889ca584'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.821583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d624ff4f9b4b0ab41435153eec5f1e', 'request_id': 'req-c8e91cac-3d13-4d4c-8665-91f1889ca584'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.832424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d624ff4f9b4b0ab41435153eec5f1e', 'request_id': 'req-c8e91cac-3d13-4d4c-8665-91f1889ca584'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.845969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] The token cache is being invalidate because project e5d624ff4f9b4b0ab41435153eec5f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:27.846166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.847376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.849179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 916/1844] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:27 2022] DELETE /identity/v3/projects/e5d624ff4f9b4b0ab41435153eec5f1e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:27.855425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:27.866386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd8f6b-b3ac-4529-9ac5-2998a88d6fa4 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:27.927786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:27.928701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:27.929527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:27.929808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] PATH_INFO: `/v3/users/a5c5e151b06b41d29058bb904e8e3595` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:27.931099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c5e151b06b41d29058bb904e8e3595)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:27.933078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:27.981145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c5e151b06b41d29058bb904e8e3595'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.981792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c5e151b06b41d29058bb904e8e3595'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.988516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c5e151b06b41d29058bb904e8e3595'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.989144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c5e151b06b41d29058bb904e8e3595'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:27.996826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-54e6552a-a142-4f71-bb3a-bdbe97dfe46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c5e151b06b41d29058bb904e8e3595'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:28.015156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 929/1845] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:27 2022] DELETE /identity/v3/users/a5c5e151b06b41d29058bb904e8e3595 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:28.339869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.341370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:28.350065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:28.358805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91cac-3d13-4d4c-8665-91f1889ca584 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.421228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.422291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.422372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.422647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] PATH_INFO: `/v3/projects/fbc5059a12f24221b2df7c88b9a75a5a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.423841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc5059a12f24221b2df7c88b9a75a5a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.426366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:28.478414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc5059a12f24221b2df7c88b9a75a5a', 'request_id': 'req-eeff3607-0e95-4b6d-85f0-40f7f8242531'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:28.484015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc5059a12f24221b2df7c88b9a75a5a', 'request_id': 'req-eeff3607-0e95-4b6d-85f0-40f7f8242531'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:28.492963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc5059a12f24221b2df7c88b9a75a5a', 'request_id': 'req-eeff3607-0e95-4b6d-85f0-40f7f8242531'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:28.504351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] The token cache is being invalidate because project fbc5059a12f24221b2df7c88b9a75a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:28.504651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:28.505333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eeff3607-0e95-4b6d-85f0-40f7f8242531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:28.506877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 917/1846] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:28 2022] DELETE /identity/v3/projects/fbc5059a12f24221b2df7c88b9a75a5a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:28.513615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.513910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.514194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.534349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:28.535634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:28.581894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:28.603192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 930/1847] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:28.611667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.611895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.612167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.633649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:28.634872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:28.642893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 918/1848] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:28.647229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.657911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3c40f-a48d-4976-b238-d052cf5b4144 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.662715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.663422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.663746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.664100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.664973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:28.665291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.666908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:28.681107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 931/1849] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:28.685553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.695442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e427f9-2f97-4779-b046-6ed0383ce8a7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.700695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.701361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.701644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.701922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.702626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.704563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:28.736348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 919/1850] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:28 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:28.738251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.749778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd97fe2-5f14-4697-83a8-cb335b43302a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.832748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.833506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.833890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.834230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.835013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.837556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:28.865941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 932/1851] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:28 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:28.867666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.878302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646f1f8-4728-4a35-83b3-898c4bf4dcbc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.884379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.885192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.885558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.885908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.886718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.888476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:28.901805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 920/1852] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:28.906329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.924208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1074551-7f4b-432d-99e9-8fc1b85b4c27 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.929892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b08b8b-323d-4921-81d0-1ae749c5d967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.930589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b08b8b-323d-4921-81d0-1ae749c5d967 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.930867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b08b8b-323d-4921-81d0-1ae749c5d967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.931146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b08b8b-323d-4921-81d0-1ae749c5d967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.931882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b08b8b-323d-4921-81d0-1ae749c5d967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.933585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b08b8b-323d-4921-81d0-1ae749c5d967 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:28.946895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 933/1853] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:28.951154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:28.961910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393638c-3c5e-469c-b11b-06d6ec7f2a6c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:28.966603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:28.967492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:28.967778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:28.968059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] PATH_INFO: `/v3/projects/c879e95c508f401fbc77e63b1e64dd16/users/6f7edee75b2845e6bbfbc64574691fb0/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:28.984597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c879e95c508f401fbc77e63b1e64dd16, user_id=6f7edee75b2845e6bbfbc64574691fb0, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:28.987280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:29.002278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 921/1854] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:28 2022] PUT /identity/v3/projects/c879e95c508f401fbc77e63b1e64dd16/users/6f7edee75b2845e6bbfbc64574691fb0/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:29.007104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aca8f-6337-42d2-9a4b-d444392f6dcb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:29.007618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aca8f-6337-42d2-9a4b-d444392f6dcb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:29.008201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aca8f-6337-42d2-9a4b-d444392f6dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:29.027909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f67aca8f-6337-42d2-9a4b-d444392f6dcb None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:29.029163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f67aca8f-6337-42d2-9a4b-d444392f6dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:29.089432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 934/1855] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:29.099325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:29.101702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:29.133281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:29.161081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351e269-217b-43c8-a0ed-b6e788e73fa4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:29.165399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:29.166093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:29.166381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:29.166663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:29.176429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:29.177890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:29.188408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 922/1856] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:29.848118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed833a9-fe1d-43f4-9bc9-9529dcb2980e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:29.849905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed833a9-fe1d-43f4-9bc9-9529dcb2980e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:29.849905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed833a9-fe1d-43f4-9bc9-9529dcb2980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:29.881844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7ed833a9-fe1d-43f4-9bc9-9529dcb2980e None None] MFA Rules not processed for user `6f7edee75b2845e6bbfbc64574691fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:29.883679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7ed833a9-fe1d-43f4-9bc9-9529dcb2980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:29.949962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:29.988246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 935/1857] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:30.023912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:30.026885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:30.089598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70022cf-70f8-4124-83ad-1167c7179276 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:30.093189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:30.093840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:30.094132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:30.094415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:30.105578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:30.107794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:30.117480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 923/1858] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:32.582001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a24d-6f5a-48cf-b537-60143c1a768a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:32.582945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a24d-6f5a-48cf-b537-60143c1a768a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:32.583405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a24d-6f5a-48cf-b537-60143c1a768a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:32.604823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2282a24d-6f5a-48cf-b537-60143c1a768a None None] MFA Rules not processed for user `6f7edee75b2845e6bbfbc64574691fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:32.606566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2282a24d-6f5a-48cf-b537-60143c1a768a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:32.620493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 936/1859] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:33.729123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:33.730191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:33.751465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da79ea0-7824-4f9f-a484-f874f03ae909 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:33.764607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:33.766284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:33.766732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:33.767171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] PATH_INFO: `/v3/users/edcad966cb144ae0a711e2589058a873` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:33.771797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcad966cb144ae0a711e2589058a873)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:33.779182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:33.892707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcad966cb144ae0a711e2589058a873'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:33.931207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcad966cb144ae0a711e2589058a873'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:33.932708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcad966cb144ae0a711e2589058a873'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:33.950691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcad966cb144ae0a711e2589058a873'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:33.951325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcad966cb144ae0a711e2589058a873'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:33.974299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 924/1860] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:33 2022] DELETE /identity/v3/users/edcad966cb144ae0a711e2589058a873 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:33.979328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:33.980204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:33.980666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.014341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:34.017087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:34.119756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:34.121540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:34.158153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 937/1861] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:34.169868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:34.175679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:34.197584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:34.222679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bc66d-31fe-4d72-a926-356cc889d4ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:34.228542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:34.229463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.229725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.230003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] PATH_INFO: `/v3/users/f641458677c5400eb5ba78b9f866bed8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.231810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f641458677c5400eb5ba78b9f866bed8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:34.233549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:34.274091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f364f-9153-47bb-bd48-cbeb10a27173 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:34.279855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151853b-d29a-4df5-bac0-c8e78e448ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:34.281269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151853b-d29a-4df5-bac0-c8e78e448ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.281562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151853b-d29a-4df5-bac0-c8e78e448ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.281847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151853b-d29a-4df5-bac0-c8e78e448ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.297728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f641458677c5400eb5ba78b9f866bed8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:34.315609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151853b-d29a-4df5-bac0-c8e78e448ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:34.317541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151853b-d29a-4df5-bac0-c8e78e448ec9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:34.326181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f641458677c5400eb5ba78b9f866bed8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:34.327352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f641458677c5400eb5ba78b9f866bed8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:34.337858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f641458677c5400eb5ba78b9f866bed8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:34.338631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f641458677c5400eb5ba78b9f866bed8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:34.351092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 938/1862] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:34.359257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 925/1863] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:34 2022] DELETE /identity/v3/users/f641458677c5400eb5ba78b9f866bed8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:34.361010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.361307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.361593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.382555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:34.383862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:34.484162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 939/1864] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:34.494541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:34.496262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:34.599279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df123b8d-b689-46e2-b88a-f331a34b5313 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:34.605764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:34.606559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:34.606945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:34.607305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:34.629813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:34.631912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:34.646191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 926/1865] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:35.253170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:35.255495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:35.273989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855773c-61e5-48b1-b6a5-7a7510be1c08 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:35.279822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:35.281058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:35.281430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:35.281769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] PATH_INFO: `/v3/projects/02f7c8a7b320491eae95111f980252ac` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:35.283120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f7c8a7b320491eae95111f980252ac)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:35.285287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:35.307671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:35.367678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f7c8a7b320491eae95111f980252ac', 'request_id': 'req-0b038f74-3924-47b1-86cb-6bf7a4016729'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.382349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f7c8a7b320491eae95111f980252ac', 'request_id': 'req-0b038f74-3924-47b1-86cb-6bf7a4016729'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.392751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f7c8a7b320491eae95111f980252ac', 'request_id': 'req-0b038f74-3924-47b1-86cb-6bf7a4016729'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.415126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] The token cache is being invalidate because project 02f7c8a7b320491eae95111f980252ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:35.415718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.416794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.419219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 940/1866] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:35 2022] DELETE /identity/v3/projects/02f7c8a7b320491eae95111f980252ac => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:35.422998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:35.439843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded96d6e-bcf3-4170-abba-4199cd509ea1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:35.440045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:35.441710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:35.599593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:35.600979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:35.601641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:35.602089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] PATH_INFO: `/v3/users/2509f223b9a74ee48166d6cf2e170b08` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:35.606113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b038f74-3924-47b1-86cb-6bf7a4016729 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:35.607789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2509f223b9a74ee48166d6cf2e170b08)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:35.610160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:35.619528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:35.621696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:35.624557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:35.624995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] PATH_INFO: `/v3/projects/f0eb112ea8a944ae8786052628ee74a9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:35.626693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0eb112ea8a944ae8786052628ee74a9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:35.628852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:35.634194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:35.696551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2509f223b9a74ee48166d6cf2e170b08'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.719555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2509f223b9a74ee48166d6cf2e170b08'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.720324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2509f223b9a74ee48166d6cf2e170b08'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.730287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2509f223b9a74ee48166d6cf2e170b08'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.731395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2509f223b9a74ee48166d6cf2e170b08'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.734831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0eb112ea8a944ae8786052628ee74a9', 'request_id': 'req-69f5c89f-9b9a-465b-87db-5463c3c73af4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.748726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0eb112ea8a944ae8786052628ee74a9', 'request_id': 'req-69f5c89f-9b9a-465b-87db-5463c3c73af4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.754255 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 927/1867] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:35 2022] DELETE /identity/v3/users/2509f223b9a74ee48166d6cf2e170b08 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:35.773654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0eb112ea8a944ae8786052628ee74a9', 'request_id': 'req-69f5c89f-9b9a-465b-87db-5463c3c73af4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.790096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] The token cache is being invalidate because project f0eb112ea8a944ae8786052628ee74a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:35.790433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.791153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.793811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 941/1868] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:35 2022] DELETE /identity/v3/projects/f0eb112ea8a944ae8786052628ee74a9 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:35.797920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:35.800220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:35.901755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19cff-2cd4-48ea-894a-9fc8f71fa4dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:35.906823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:35.907750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:35.908088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:35.908503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] PATH_INFO: `/v3/users/6bc8261177cb4fa5b5e8157f5ef3ec45` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:35.910626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc8261177cb4fa5b5e8157f5ef3ec45)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:35.913103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:35.967798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc8261177cb4fa5b5e8157f5ef3ec45'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.992176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc8261177cb4fa5b5e8157f5ef3ec45'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:35.993527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc8261177cb4fa5b5e8157f5ef3ec45'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.003287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc8261177cb4fa5b5e8157f5ef3ec45'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.003992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc8261177cb4fa5b5e8157f5ef3ec45'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.018662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 928/1869] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:35 2022] DELETE /identity/v3/users/6bc8261177cb4fa5b5e8157f5ef3ec45 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:36.166800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.168774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:36.221925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:36.290506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5c89f-9b9a-465b-87db-5463c3c73af4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.370217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.371244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.371717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.372034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] PATH_INFO: `/v3/projects/484ddde9dfe34888b27ce2cd1a550f49` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.373377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=484ddde9dfe34888b27ce2cd1a550f49)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.375348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.398353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.399871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:36.420678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f587-480d-425f-88af-7b484b9c303a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.426451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.428198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.429321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.429705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] PATH_INFO: `/v3/projects/5ce6bf92cca04078814abd290002f098` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.431245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce6bf92cca04078814abd290002f098)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.433713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.436952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:36.456268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484ddde9dfe34888b27ce2cd1a550f49', 'request_id': 'req-26d8b090-4b40-42a8-be87-d2e2526e22b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.468006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484ddde9dfe34888b27ce2cd1a550f49', 'request_id': 'req-26d8b090-4b40-42a8-be87-d2e2526e22b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.479993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484ddde9dfe34888b27ce2cd1a550f49', 'request_id': 'req-26d8b090-4b40-42a8-be87-d2e2526e22b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.493165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] The token cache is being invalidate because project 484ddde9dfe34888b27ce2cd1a550f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:36.493459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.494140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-26d8b090-4b40-42a8-be87-d2e2526e22b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.495767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 942/1870] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:36 2022] DELETE /identity/v3/projects/484ddde9dfe34888b27ce2cd1a550f49 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:36.500673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cfa01-6994-4681-bb91-6daf9cbe7db8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.500995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cfa01-6994-4681-bb91-6daf9cbe7db8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.501241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cfa01-6994-4681-bb91-6daf9cbe7db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.505369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce6bf92cca04078814abd290002f098', 'request_id': 'req-25628f65-56b9-4ee0-9a9b-96bb76428f2a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.515577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce6bf92cca04078814abd290002f098', 'request_id': 'req-25628f65-56b9-4ee0-9a9b-96bb76428f2a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.521137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-073cfa01-6994-4681-bb91-6daf9cbe7db8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:36.522432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-073cfa01-6994-4681-bb91-6daf9cbe7db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:36.524668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce6bf92cca04078814abd290002f098', 'request_id': 'req-25628f65-56b9-4ee0-9a9b-96bb76428f2a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.539238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] The token cache is being invalidate because project 5ce6bf92cca04078814abd290002f098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:36.539533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.540192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.541818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 929/1871] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:36 2022] DELETE /identity/v3/projects/5ce6bf92cca04078814abd290002f098 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:36.545695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.547087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:36.587469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 943/1872] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:36.595451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.595752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.596040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.614664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25628f65-56b9-4ee0-9a9b-96bb76428f2a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.614889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:36.615891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:36.619667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.620602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.620905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.621156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] PATH_INFO: `/v3/users/4460e5f9d413402da059a6d8d84c4aba` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.622527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4460e5f9d413402da059a6d8d84c4aba)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.624568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.625273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 944/1873] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:36.629959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.635501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:36.639415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba92bc4-e45d-4e31-bc95-f458cc15fdfd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.645430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.646103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.646707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.647133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.648736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:36.649081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.651147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.666280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 945/1874] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:36.671199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.683247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d556942-74a3-4e8d-8e69-ffb27a989d80 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.683546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4460e5f9d413402da059a6d8d84c4aba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.690911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.691821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.692261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.692896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.694975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.697309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.707341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4460e5f9d413402da059a6d8d84c4aba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.708182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4460e5f9d413402da059a6d8d84c4aba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.717054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4460e5f9d413402da059a6d8d84c4aba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.717799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4460e5f9d413402da059a6d8d84c4aba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:36.733730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 946/1875] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:36 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:36.740078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.743789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:36.744419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 930/1876] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:36 2022] DELETE /identity/v3/users/4460e5f9d413402da059a6d8d84c4aba => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:36.823638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d0a0b-7766-4c6b-ae3e-486409ee06b8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.892134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.892854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.893142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.893419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.894121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.896094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.918071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 947/1877] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:36 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:36.922293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.935289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54719ad-2631-4136-aff7-f776317d912f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.941495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.942607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.943569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:36.943861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:36.945275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:36.946924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:36.950009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:36.973227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 931/1878] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:36.977988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:36.990615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73128c-61d2-4714-9731-db7f12ea3c34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:36.997003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbfd4a-f0e2-49bd-b1b9-4f966335a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:36.998042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbfd4a-f0e2-49bd-b1b9-4f966335a4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:36.998407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbfd4a-f0e2-49bd-b1b9-4f966335a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.000465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbfd4a-f0e2-49bd-b1b9-4f966335a4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.000465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbfd4a-f0e2-49bd-b1b9-4f966335a4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.001826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbfd4a-f0e2-49bd-b1b9-4f966335a4e7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.016376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 948/1879] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:37.018678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.032748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a5a5d-c89f-49a3-a8a9-7327a353b927 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.040133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.041319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.041685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.042030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] PATH_INFO: `/v3/projects/4549abda210b4fa8baaf75edbb7b250a/users/c7878c9eae064f4cbeb6a66712465a9d/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.069881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4549abda210b4fa8baaf75edbb7b250a, user_id=c7878c9eae064f4cbeb6a66712465a9d, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.074885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.099121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 932/1880] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:37 2022] PUT /identity/v3/projects/4549abda210b4fa8baaf75edbb7b250a/users/c7878c9eae064f4cbeb6a66712465a9d/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:37.110244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.110539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.110825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.139343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] MFA Rules not processed for user `c7878c9eae064f4cbeb6a66712465a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:37.140639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.187440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:37.194509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.196339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.224807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 949/1881] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:37.230020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.269243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c541740-86f1-427c-956e-bdece09f159b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.274460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.275379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.275663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.276048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] PATH_INFO: `/v3/projects/baf8082dddc2410b8242f99b7fe6f27b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.277501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf8082dddc2410b8242f99b7fe6f27b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.279608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.303026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f678-dcc6-410e-adc6-0e30c4e4d386 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.310146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.311120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.311637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.311963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.312767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.314994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.339378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf8082dddc2410b8242f99b7fe6f27b', 'request_id': 'req-4d863d1d-6dae-4c04-be0e-a065f8ef2362'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:37.346800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf8082dddc2410b8242f99b7fe6f27b', 'request_id': 'req-4d863d1d-6dae-4c04-be0e-a065f8ef2362'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:37.349172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 950/1882] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:37.354340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.360922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf8082dddc2410b8242f99b7fe6f27b', 'request_id': 'req-4d863d1d-6dae-4c04-be0e-a065f8ef2362'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:37.374385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] The token cache is being invalidate because project baf8082dddc2410b8242f99b7fe6f27b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:37.374703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:37.375373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d863d1d-6dae-4c04-be0e-a065f8ef2362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:37.377273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 933/1883] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:37 2022] DELETE /identity/v3/projects/baf8082dddc2410b8242f99b7fe6f27b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:37.383679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af14da-b9fa-4131-a342-d96b2cd8174e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.384114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af14da-b9fa-4131-a342-d96b2cd8174e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.384498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af14da-b9fa-4131-a342-d96b2cd8174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.413111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-07af14da-b9fa-4131-a342-d96b2cd8174e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:37.414434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07af14da-b9fa-4131-a342-d96b2cd8174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.425681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1f5a-70b7-464b-9f0d-dab0b91a61cd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.432534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 934/1884] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:37.433593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.434692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.435154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.435611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.436898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.439590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.439920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.440040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.440172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.462182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:37.463427 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.474066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 951/1885] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:37.474248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 935/1886] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:37.475238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.476551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.477601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.486032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f60480-b4d2-440e-90d8-56f43468aa3f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.489963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6565db8-c15e-4aca-a0cb-b2c7329c53ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.492547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.493218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.493508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.493791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.494520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:37.494834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.495045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.495714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.495997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.496275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.496901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.497056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.498620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.509851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 936/1887] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:37.513985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 952/1888] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:37.514393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.516590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.526288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02c2d-bd09-4fb9-8355-d0608094810d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.529081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daa1ef-e383-42d7-9f73-96af8b7e016a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.531491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.532168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.532589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.533052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.534423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.535475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.536414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.536542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.536810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.537017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] PATH_INFO: `/v3/projects/83ed7762833747b88ae670dd4d2e2709/users/01aaeaad3bf84a7893453d321bc93e70/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.562643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ed7762833747b88ae670dd4d2e2709, user_id=01aaeaad3bf84a7893453d321bc93e70, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.565829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.571211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 937/1889] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:37.576460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.582130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 953/1890] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:37 2022] PUT /identity/v3/projects/83ed7762833747b88ae670dd4d2e2709/users/01aaeaad3bf84a7893453d321bc93e70/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:37.586097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.601053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:37.619713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:37.662562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a5390-e0bd-44ec-a35e-c63e0fc96eb7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.666792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6458-6209-4c4f-9c3e-b18dd9e10fac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.668970 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.669658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.670066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.670299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.671165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.672854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.681575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.682339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.682697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.683096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.683696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 954/1891] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:37.684074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.685999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.687822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.700704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa9811-a958-4ae9-9229-d3743dd49511 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.705990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb01d7-cc1f-40ed-b086-c6eb47d53b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.706942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb01d7-cc1f-40ed-b086-c6eb47d53b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.707252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb01d7-cc1f-40ed-b086-c6eb47d53b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.707515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb01d7-cc1f-40ed-b086-c6eb47d53b32 admin admin] PATH_INFO: `/v3/projects/83ed7762833747b88ae670dd4d2e2709/users/01aaeaad3bf84a7893453d321bc93e70/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.711008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 938/1892] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:37.712135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.713163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb01d7-cc1f-40ed-b086-c6eb47d53b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ed7762833747b88ae670dd4d2e2709, user_id=01aaeaad3bf84a7893453d321bc93e70, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.716259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb01d7-cc1f-40ed-b086-c6eb47d53b32 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.722613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3ad97-5ab0-410a-85cc-e7f8a75bb5a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.730324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 955/1893] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:37 2022] PUT /identity/v3/projects/83ed7762833747b88ae670dd4d2e2709/users/01aaeaad3bf84a7893453d321bc93e70/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:37.741480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.741905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.742275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.769021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] MFA Rules not processed for user `01aaeaad3bf84a7893453d321bc93e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:37.770243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.786348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.787027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.787317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.787600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.788586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.790222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.803011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 939/1894] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:37.805373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.816009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c813-5e86-4798-a163-576ca8aa3b0f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.821517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.822188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.822476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.823002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.823719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.825358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.835643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 940/1895] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:45:37.839885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.840135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 956/1896] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:37.851119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8f9bc-f377-4820-9401-d83632d797c4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.851750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:37.853228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.856407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f0228-5fa8-42da-bc86-127aea13d5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.857299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f0228-5fa8-42da-bc86-127aea13d5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.857587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f0228-5fa8-42da-bc86-127aea13d5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.857870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f0228-5fa8-42da-bc86-127aea13d5d4 admin admin] PATH_INFO: `/v3/projects/253ac2630dfd4869af300b12c70ab051/users/69aa24e6fe704102af73817a11872512/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.874384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f0228-5fa8-42da-bc86-127aea13d5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253ac2630dfd4869af300b12c70ab051, user_id=69aa24e6fe704102af73817a11872512, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:37.877809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f0228-5fa8-42da-bc86-127aea13d5d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:37.891868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 941/1897] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:37 2022] PUT /identity/v3/projects/253ac2630dfd4869af300b12c70ab051/users/69aa24e6fe704102af73817a11872512/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:37.904113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.904465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.904757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.912398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7779a-8da3-474c-a18e-d9e4c8a52503 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:37.934623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] MFA Rules not processed for user `69aa24e6fe704102af73817a11872512`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:37.935999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:37.959086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:37.964983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:37.965658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:37.966731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:37.967121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:37.998396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:38.012050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 942/1898] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:38.017338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.047433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.048984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.068455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 957/1899] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:45:38.100033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.122612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d4923-4169-4c5b-ad05-f4429ce817a1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.128061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.128782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.129066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.129346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.130048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.132592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.181415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 943/1900] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:38 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:38.181415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.187666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bf95-4e98-4f71-b1c8-b9c8336cfa6a service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.247303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd474d-2d79-4a67-bf97-f32d941390ca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.254021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.255132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.255775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.256068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.256906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.259309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.264711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.265597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.265881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.266154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] PATH_INFO: `/v3/users/c7878c9eae064f4cbeb6a66712465a9d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.267457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7878c9eae064f4cbeb6a66712465a9d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.269436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.282884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 944/1901] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:38 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:38.286750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.300465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c6e3-7c36-4ea2-8f45-45956b7edfb8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.308325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.309033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.309343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.309601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.310362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.312182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.324317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 945/1902] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:45:38.329004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.339806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7878c9eae064f4cbeb6a66712465a9d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:38.341885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7878c9eae064f4cbeb6a66712465a9d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:38.343281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:38.346973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b5df-3b57-4d8c-9529-e2c300e5b329 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.359189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.359834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7878c9eae064f4cbeb6a66712465a9d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:38.361147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.361428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.361708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] PATH_INFO: `/v3/projects/1d647b38444c41c9b6fa20c25c5b7e68/users/dd6962a8fa7142c0ad08381b6c8d7479/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.363224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7878c9eae064f4cbeb6a66712465a9d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:38.374508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebb6189d-a582-469b-ab12-8156e6e094f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7878c9eae064f4cbeb6a66712465a9d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:38.386001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d647b38444c41c9b6fa20c25c5b7e68, user_id=dd6962a8fa7142c0ad08381b6c8d7479, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.390246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.411976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 958/1903] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:38 2022] DELETE /identity/v3/users/c7878c9eae064f4cbeb6a66712465a9d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:38.413605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 946/1904] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:38 2022] PUT /identity/v3/projects/1d647b38444c41c9b6fa20c25c5b7e68/users/dd6962a8fa7142c0ad08381b6c8d7479/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:38.418954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.419156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.419339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.419629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.421121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:38.448665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:38.451695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:38.521103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:38.572212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 959/1905] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:38.582841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.584323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:38.601077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a9921-a0c9-4651-aa09-9990c0dcc087 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.611459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.613167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.613724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.614231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.614983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.617425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.631017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 947/1906] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:38.636498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.651690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c709ed-61e7-40eb-a6c5-b60f4f14a086 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.658964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ac96d-55f3-40b7-8b16-59dc0463f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.659988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac96d-55f3-40b7-8b16-59dc0463f672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.660213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac96d-55f3-40b7-8b16-59dc0463f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.660476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac96d-55f3-40b7-8b16-59dc0463f672 admin admin] PATH_INFO: `/v3/projects/1d647b38444c41c9b6fa20c25c5b7e68/users/dd6962a8fa7142c0ad08381b6c8d7479/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.662673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ac96d-55f3-40b7-8b16-59dc0463f672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d647b38444c41c9b6fa20c25c5b7e68, user_id=dd6962a8fa7142c0ad08381b6c8d7479, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.667250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ac96d-55f3-40b7-8b16-59dc0463f672 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.668664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b56684-7951-4987-bf5d-767a51c85128 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.672791 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:38.673556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.673905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.674370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.680940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 948/1907] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:38 2022] PUT /identity/v3/projects/1d647b38444c41c9b6fa20c25c5b7e68/users/dd6962a8fa7142c0ad08381b6c8d7479/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:38.687818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:38.689354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:38.696757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:38.697451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:38.698759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:38.707920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 960/1908] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:45:38.732968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] MFA Rules not processed for user `dd6962a8fa7142c0ad08381b6c8d7479`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:38.747208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:38.855199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 949/1909] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:38.870724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:38.872280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:38.962425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4ebe2-3d45-4996-a518-4ebe55ef8d36 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:38.987144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:39.024572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.025515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.026161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.026161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.036904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.038802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.050571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 961/1910] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:39.096851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.099663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:39.129838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.132878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:39.153249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b857ba9-3d16-4f62-a6ca-1c32c0a477f0 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.159646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.161389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.161757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.162210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.166759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:39.242169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2d6dd-64df-4e7a-bbc0-84707f5a468f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.249805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.251243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.251799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.252371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] PATH_INFO: `/v3/projects/4549abda210b4fa8baaf75edbb7b250a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.254474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4549abda210b4fa8baaf75edbb7b250a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.258424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.310604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.313327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.332623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 962/1911] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:39.354931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:39.361160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4549abda210b4fa8baaf75edbb7b250a', 'request_id': 'req-937e8950-e8c8-4be2-9038-212160f3df73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.378127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4549abda210b4fa8baaf75edbb7b250a', 'request_id': 'req-937e8950-e8c8-4be2-9038-212160f3df73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.395137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4549abda210b4fa8baaf75edbb7b250a', 'request_id': 'req-937e8950-e8c8-4be2-9038-212160f3df73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.427539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] The token cache is being invalidate because project 4549abda210b4fa8baaf75edbb7b250a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:39.428221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.429567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.432531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 950/1912] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:39 2022] DELETE /identity/v3/projects/4549abda210b4fa8baaf75edbb7b250a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:39.436230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:39.450497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450a69-4b44-40e7-8e43-a03873ae86af service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:39.548613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:39.550121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:39.550420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:39.550703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] PATH_INFO: `/v3/users/01aaeaad3bf84a7893453d321bc93e70` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:39.553905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01aaeaad3bf84a7893453d321bc93e70)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:39.573198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:39.653813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01aaeaad3bf84a7893453d321bc93e70'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.654587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01aaeaad3bf84a7893453d321bc93e70'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.689828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01aaeaad3bf84a7893453d321bc93e70'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.691562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01aaeaad3bf84a7893453d321bc93e70'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.708900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-086f546a-b5f2-4561-af29-2722a5429cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01aaeaad3bf84a7893453d321bc93e70'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:39.722818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:39.796512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 963/1913] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:39 2022] DELETE /identity/v3/users/01aaeaad3bf84a7893453d321bc93e70 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:40.874840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:40.876453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:40.911372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e8950-e8c8-4be2-9038-212160f3df73 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:40.982244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:40.983252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:40.983419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:40.983699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] PATH_INFO: `/v3/projects/83ed7762833747b88ae670dd4d2e2709` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:40.985387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ed7762833747b88ae670dd4d2e2709)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:40.987462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.043308 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ed7762833747b88ae670dd4d2e2709', 'request_id': 'req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:41.049828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ed7762833747b88ae670dd4d2e2709', 'request_id': 'req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:41.059043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ed7762833747b88ae670dd4d2e2709', 'request_id': 'req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:41.072401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] The token cache is being invalidate because project 83ed7762833747b88ae670dd4d2e2709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:41.072946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:41.073653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70d0c660-a7b0-43c4-bbb3-c8690d6aa502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:41.075200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 951/1914] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:40 2022] DELETE /identity/v3/projects/83ed7762833747b88ae670dd4d2e2709 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:41.080165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.080743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.080743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.099656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:41.100623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:41.166159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 964/1915] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:41.173878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.174174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.174456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.194250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:41.195486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:41.203989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 952/1916] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:41.208123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.218460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9e29d-d0b1-42a2-889a-828a24ef3717 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.223146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.223822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.224113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.224473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.225191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:41.225622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.227318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.247123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 965/1917] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:41.249488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.260055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad6127-774a-4a34-a546-078a6278b8d1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.266777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.267759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.268173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.268590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.269589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.271848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.303918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 953/1918] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:41 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:41.308835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.324535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896c24c-27b1-400c-9c90-28d4f7f7e941 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.404057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.404997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.405355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.405707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.406678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.408840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.446603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 966/1919] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:41 2022] POST /identity/v3/users => generated 305 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:41.448237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.464846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffebbf2-0a04-45c2-82b6-541f4cfd3919 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.471859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.473811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.473811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.476459 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.477533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.479594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.498552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 954/1920] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:41.500219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.514186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eefe51-def6-4c84-905e-5dac4271f262 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.529464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd8a4-dfd0-4af4-8418-b9b19ae1c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.530701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd8a4-dfd0-4af4-8418-b9b19ae1c022 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.530989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd8a4-dfd0-4af4-8418-b9b19ae1c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.533100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd8a4-dfd0-4af4-8418-b9b19ae1c022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.535688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbd8a4-dfd0-4af4-8418-b9b19ae1c022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.539427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbd8a4-dfd0-4af4-8418-b9b19ae1c022 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.563678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.563798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 967/1921] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:45:41.591160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e13d5-8f73-416c-bce6-276fa3f924c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.595968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.597001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.597317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.597598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] PATH_INFO: `/v3/projects/be3a448c44af40d9b0297a48948f7f38/users/7784bf417d614947a8632e5b00805c52/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.615041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3a448c44af40d9b0297a48948f7f38, user_id=7784bf417d614947a8632e5b00805c52, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.618276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:41.641121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:41.651918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 955/1922] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:41 2022] PUT /identity/v3/projects/be3a448c44af40d9b0297a48948f7f38/users/7784bf417d614947a8632e5b00805c52/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:41.663216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.663790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.664208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.699900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] MFA Rules not processed for user `7784bf417d614947a8632e5b00805c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:41.701622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:41.797360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:41.854098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 968/1923] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:41.858215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:41.947332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661782dd-4c5d-4f94-8cd6-54c373342f41 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:41.955138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:41.956929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:41.957261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:41.957557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:41.958743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:41.962076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.000452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 956/1924] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:41 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:42.004701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.020262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440386-370d-4821-99cb-ba1e287db423 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.096966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.097677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.097967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.098250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.098944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.101628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.122855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 969/1925] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:42 2022] POST /identity/v3/users => generated 309 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:42.127056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.138173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96f96a-2946-4594-b6b7-14edb52c5ad6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.142916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.143591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.143882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.144164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.144924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.146671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.158804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 957/1926] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:42.163527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.174195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab1b63-c49f-4cb5-a330-f3b832bcdad6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.178975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.179917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.180228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.180548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] PATH_INFO: `/v3/projects/d643bbf093804b96b3b2a5cad6a240be/users/38b6695ef3bc494ea62a067fc3a438c9/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.196241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d643bbf093804b96b3b2a5cad6a240be, user_id=38b6695ef3bc494ea62a067fc3a438c9, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.199199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.213050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 970/1927] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:42 2022] PUT /identity/v3/projects/d643bbf093804b96b3b2a5cad6a240be/users/38b6695ef3bc494ea62a067fc3a438c9/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:42.217149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.227047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b8ae-039c-4102-af58-7ccec8fe3164 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.282755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1c24-8008-4d1d-a4d2-55eb8bfaec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.283415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1c24-8008-4d1d-a4d2-55eb8bfaec17 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.283695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1c24-8008-4d1d-a4d2-55eb8bfaec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.283974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1c24-8008-4d1d-a4d2-55eb8bfaec17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.284788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1c24-8008-4d1d-a4d2-55eb8bfaec17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.286381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1c24-8008-4d1d-a4d2-55eb8bfaec17 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.296171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 958/1928] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:42.300408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.311341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52326f3-03d0-490d-94f6-0c1b75c6727a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.316675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.317562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.317850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.318149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] PATH_INFO: `/v3/projects/d643bbf093804b96b3b2a5cad6a240be/users/38b6695ef3bc494ea62a067fc3a438c9/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.320875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d643bbf093804b96b3b2a5cad6a240be, user_id=38b6695ef3bc494ea62a067fc3a438c9, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.323813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.335693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 971/1929] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:42 2022] PUT /identity/v3/projects/d643bbf093804b96b3b2a5cad6a240be/users/38b6695ef3bc494ea62a067fc3a438c9/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:42.347236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.347614 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.347930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.376733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] MFA Rules not processed for user `38b6695ef3bc494ea62a067fc3a438c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:42.377873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:42.435150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:42.444836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 959/1930] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:42.455907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.457618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:42.517083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc6ffa-e87b-4313-9a35-d560a9b982b4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.520752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.521402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.521690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.522010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.531167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.532680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.542950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 972/1931] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:42.727045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:42.728604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:42.740672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8053829-54e4-40bf-ac26-cad859797d80 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:42.744439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b620afd-29fa-4275-8f5f-8926d668216d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:42.745102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620afd-29fa-4275-8f5f-8926d668216d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:42.745394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620afd-29fa-4275-8f5f-8926d668216d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:42.745678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620afd-29fa-4275-8f5f-8926d668216d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:42.808009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b620afd-29fa-4275-8f5f-8926d668216d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:42.809486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b620afd-29fa-4275-8f5f-8926d668216d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:42.820687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 960/1932] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:49.661859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:49.663486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:49.706672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:49.733802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0677f0-de76-46c4-849f-dcca23406e63 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:49.738609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:49.739495 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:49.739786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:49.740069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] PATH_INFO: `/v3/users/69aa24e6fe704102af73817a11872512` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:49.742151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69aa24e6fe704102af73817a11872512)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:49.744119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:49.797421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69aa24e6fe704102af73817a11872512'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:49.798356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69aa24e6fe704102af73817a11872512'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:49.808126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69aa24e6fe704102af73817a11872512'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:49.809149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69aa24e6fe704102af73817a11872512'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:49.818031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69aa24e6fe704102af73817a11872512'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:49.837629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 973/1933] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:49 2022] DELETE /identity/v3/users/69aa24e6fe704102af73817a11872512 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:49.842191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:49.842933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:49.842933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:49.861927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:49.863245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:49.936371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 961/1934] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:49.943328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:49.944917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:50.003253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5560cd-7908-435f-bcc1-3ca7263f9170 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.007377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.008046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.008358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.008645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.018065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.019507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.029626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 974/1935] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:50.319128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.320873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:50.353781 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161cc59-f8da-42ee-8442-88d41dafd877 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.359270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.360219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.360613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.360913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] PATH_INFO: `/v3/projects/253ac2630dfd4869af300b12c70ab051` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.362112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253ac2630dfd4869af300b12c70ab051)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.364061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.415565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253ac2630dfd4869af300b12c70ab051', 'request_id': 'req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.421031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253ac2630dfd4869af300b12c70ab051', 'request_id': 'req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.434010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253ac2630dfd4869af300b12c70ab051', 'request_id': 'req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.449184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] The token cache is being invalidate because project 253ac2630dfd4869af300b12c70ab051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:50.449390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.450075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.452401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 962/1936] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:50 2022] DELETE /identity/v3/projects/253ac2630dfd4869af300b12c70ab051 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:50.456200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.468691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5902ab8-29e8-4104-9728-8a84a61f39ea service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.528620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:50.538631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:50.539549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:50.540009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:50.540677 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] PATH_INFO: `/v3/users/dd6962a8fa7142c0ad08381b6c8d7479` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:50.542077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6962a8fa7142c0ad08381b6c8d7479)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:50.544080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:50.600636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6962a8fa7142c0ad08381b6c8d7479'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.601400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6962a8fa7142c0ad08381b6c8d7479'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.609485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6962a8fa7142c0ad08381b6c8d7479'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.610218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6962a8fa7142c0ad08381b6c8d7479'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.620674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc3a5a31-7415-433f-ace1-1b7b6e00fd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6962a8fa7142c0ad08381b6c8d7479'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:50.640020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 975/1937] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:50 2022] DELETE /identity/v3/users/dd6962a8fa7142c0ad08381b6c8d7479 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:50.955674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:50.957215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:50.971780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a723c7-5f43-4b56-a068-f4ab4c42f0d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:50.990017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:51.071235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.072391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.072953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.073216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] PATH_INFO: `/v3/projects/1d647b38444c41c9b6fa20c25c5b7e68` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.074489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d647b38444c41c9b6fa20c25c5b7e68)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.076481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.126663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d647b38444c41c9b6fa20c25c5b7e68', 'request_id': 'req-9e23b2aa-f659-49ae-afc9-07712b0711f8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:51.133302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d647b38444c41c9b6fa20c25c5b7e68', 'request_id': 'req-9e23b2aa-f659-49ae-afc9-07712b0711f8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:51.142248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d647b38444c41c9b6fa20c25c5b7e68', 'request_id': 'req-9e23b2aa-f659-49ae-afc9-07712b0711f8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:51.154616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] The token cache is being invalidate because project 1d647b38444c41c9b6fa20c25c5b7e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:51.154924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:51.155588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e23b2aa-f659-49ae-afc9-07712b0711f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:51.157178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 963/1938] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:50 2022] DELETE /identity/v3/projects/1d647b38444c41c9b6fa20c25c5b7e68 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:51.162092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.162711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.162940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.183004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:51.184265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:51.247666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 976/1939] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:51.255669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.256008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.256274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.276972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:51.278189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:51.286600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 964/1940] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:51.291646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.301512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25c53-b44d-418c-abfd-202417cbbd72 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.307221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.307960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.308328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.308690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.309512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:51.309888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.311558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.328609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 977/1941] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:45:51.334143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.344831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f19726-f824-456c-8a05-528e8bb72bbb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.350801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.352025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.352681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.353196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.354376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.357590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.395816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 965/1942] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:45:51.400073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.411558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da257435-c305-4509-88d4-242c41b7f604 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.485000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.485000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.485000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.485000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.485485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.485485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.491821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:51.509161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 978/1943] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:51.517540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.530328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f501e0-f111-49c8-9725-2541a7f7aa53 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.535431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.536222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.536672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.537880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.539267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.541421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.556126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 966/1944] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:51.561811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.573302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261cd1d-c37c-42b1-9be6-daee44c50611 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.581173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5e11f-e1a9-41f0-be67-94f70c64329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.581959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5e11f-e1a9-41f0-be67-94f70c64329e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.582244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5e11f-e1a9-41f0-be67-94f70c64329e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.582520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5e11f-e1a9-41f0-be67-94f70c64329e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.583250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5e11f-e1a9-41f0-be67-94f70c64329e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.585831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5e11f-e1a9-41f0-be67-94f70c64329e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.603667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 979/1945] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:51.604924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.618328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08669964-4523-417c-af70-7d5222cb6ecd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.624747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.625647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.625927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.626205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] PATH_INFO: `/v3/projects/66580b18e5c54b3f9ecf199e046f3cd2/users/13ef43517ff74c21a5fd4bb95a0c52f2/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.641740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66580b18e5c54b3f9ecf199e046f3cd2, user_id=13ef43517ff74c21a5fd4bb95a0c52f2, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.644428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.658610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 967/1946] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:51 2022] PUT /identity/v3/projects/66580b18e5c54b3f9ecf199e046f3cd2/users/13ef43517ff74c21a5fd4bb95a0c52f2/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:51.669724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.670010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.670296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.698229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] MFA Rules not processed for user `13ef43517ff74c21a5fd4bb95a0c52f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:51.699473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:51.779972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 980/1947] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:51.781096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.835055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:51.857761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db18fa-4112-44a1-b293-b882adc86712 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.869380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.869380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.869674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.869836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.870502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.872627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:51.910491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 968/1948] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/projects => generated 399 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:51.915226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:51.927772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e20cb-1046-48eb-a9ed-9ccbbe80cba8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:51.994421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:51.995185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:51.995478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:51.995787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:51.996628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:51.998722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.021042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 981/1949] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:51 2022] POST /identity/v3/users => generated 306 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:52.026009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:52.036268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294caee-0de1-4af9-b3d1-ef11268ca375 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:52.041775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:52.042144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.042432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.042714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.043437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:52.045202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.056743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 969/1950] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:52.062795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:52.076596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca219-c83d-4153-90fa-73538c1764a0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:52.081463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:52.082371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.082657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.082938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] PATH_INFO: `/v3/projects/62d6c66d76ec4d3e8b7d41a21ec1e34a/users/578d7de9e7aa43ee9bcc5216418126ee/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.101354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d6c66d76ec4d3e8b7d41a21ec1e34a, user_id=578d7de9e7aa43ee9bcc5216418126ee, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:52.105158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.121148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 982/1951] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:52 2022] PUT /identity/v3/projects/62d6c66d76ec4d3e8b7d41a21ec1e34a/users/578d7de9e7aa43ee9bcc5216418126ee/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:52.125289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:52.136143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872ba34-05be-4f0c-ac59-280a1a2cb7a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:52.192523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e315c-be84-46dd-9054-84f5c2d94217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:52.193875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e315c-be84-46dd-9054-84f5c2d94217 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.194274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e315c-be84-46dd-9054-84f5c2d94217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.194561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e315c-be84-46dd-9054-84f5c2d94217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.195347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e315c-be84-46dd-9054-84f5c2d94217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:52.197125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e315c-be84-46dd-9054-84f5c2d94217 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.207626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 970/1952] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:52.212476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:52.223098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c410d9-edf7-4b01-b79a-dfcacfcf9faf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:52.228146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:52.229122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.229409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.229686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] PATH_INFO: `/v3/projects/62d6c66d76ec4d3e8b7d41a21ec1e34a/users/578d7de9e7aa43ee9bcc5216418126ee/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.231648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d6c66d76ec4d3e8b7d41a21ec1e34a, user_id=578d7de9e7aa43ee9bcc5216418126ee, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:52.235084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.246629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 983/1953] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:52 2022] PUT /identity/v3/projects/62d6c66d76ec4d3e8b7d41a21ec1e34a/users/578d7de9e7aa43ee9bcc5216418126ee/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:52.258526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.259464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.259464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.295047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] MFA Rules not processed for user `578d7de9e7aa43ee9bcc5216418126ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:52.296381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:52.378484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 971/1954] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:52.391467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:52.393976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:52.438389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:52.470330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a1e81-256d-439d-9819-4642bb449c72 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:52.474101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:52.475569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.475957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.476249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.486393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:52.487940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.499528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 984/1955] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:52.594358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:52.596253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:52.629094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223464fb-3b5a-4ac8-ab5d-08126aaccd76 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:52.633470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea462452-0039-469f-82b3-d946e0105cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:52.634138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462452-0039-469f-82b3-d946e0105cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:52.634420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462452-0039-469f-82b3-d946e0105cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:52.634699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462452-0039-469f-82b3-d946e0105cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:52.680496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:52.701936 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea462452-0039-469f-82b3-d946e0105cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:52.703405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea462452-0039-469f-82b3-d946e0105cd2 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:52.713754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 972/1956] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:55.795130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:55.796672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:55.883774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738469-8860-47a4-84e2-d4bdb017a1ba service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:55.889305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:55.890333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.890619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.890898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] PATH_INFO: `/v3/users/4433005759624d9ab32457f753d07b11` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:55.892116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4433005759624d9ab32457f753d07b11)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:55.894098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:55.952726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4433005759624d9ab32457f753d07b11'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:55.953421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4433005759624d9ab32457f753d07b11'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:55.962027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4433005759624d9ab32457f753d07b11'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:55.962904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4433005759624d9ab32457f753d07b11'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:55.972974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4433005759624d9ab32457f753d07b11'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:55.992025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 985/1957] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:55 2022] DELETE /identity/v3/users/4433005759624d9ab32457f753d07b11 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:55.996703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:55.997348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:55.997348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:56.014534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:56.015729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:56.076885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 973/1958] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:56.087396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:56.089160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:56.145984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:56.153539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289b7b-315d-484d-9db9-aa066a8061ca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:56.157884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:56.158552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:56.158973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:56.159384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:56.170041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:56.170713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:56.181582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 986/1959] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:56.466301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:56.468840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:56.494366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d75bd-56ce-484f-8d64-5eaddea785fc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:56.501167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:56.502131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:56.502646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:56.502833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] PATH_INFO: `/v3/projects/0484da23a9ab4d22aa654f0874f1b632` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:56.504215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0484da23a9ab4d22aa654f0874f1b632)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:56.507708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:56.588436 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0484da23a9ab4d22aa654f0874f1b632', 'request_id': 'req-8eb20af0-de7d-4286-bb59-0340f7d84931'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.595890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0484da23a9ab4d22aa654f0874f1b632', 'request_id': 'req-8eb20af0-de7d-4286-bb59-0340f7d84931'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.605119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0484da23a9ab4d22aa654f0874f1b632', 'request_id': 'req-8eb20af0-de7d-4286-bb59-0340f7d84931'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.624482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] The token cache is being invalidate because project 0484da23a9ab4d22aa654f0874f1b632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:56.625077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.625970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.631947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 974/1960] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:56 2022] DELETE /identity/v3/projects/0484da23a9ab4d22aa654f0874f1b632 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:56.632894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:56.643158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563d27-6581-4d29-bf9a-85eff038a505 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:56.725231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:56.726486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:56.726832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:56.727184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] PATH_INFO: `/v3/users/01a6117f9d5948a1940547079b1904f6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:56.729114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a6117f9d5948a1940547079b1904f6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:56.731694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:56.801313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a6117f9d5948a1940547079b1904f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.811150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a6117f9d5948a1940547079b1904f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.821318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a6117f9d5948a1940547079b1904f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.822144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a6117f9d5948a1940547079b1904f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.831905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20b5efcc-c372-4ad7-9b69-0c4a9f509b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a6117f9d5948a1940547079b1904f6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:56.853324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 987/1961] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:56 2022] DELETE /identity/v3/users/01a6117f9d5948a1940547079b1904f6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:57.188832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.190478 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:57.205060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb20af0-de7d-4286-bb59-0340f7d84931 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.269913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.270786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.271179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.271548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] PATH_INFO: `/v3/projects/e536c0e1e08340e89b27fd10934f4843` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.272898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e536c0e1e08340e89b27fd10934f4843)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.274896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.278543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:57.375605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e536c0e1e08340e89b27fd10934f4843', 'request_id': 'req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.384872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e536c0e1e08340e89b27fd10934f4843', 'request_id': 'req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.396700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e536c0e1e08340e89b27fd10934f4843', 'request_id': 'req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.416726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] The token cache is being invalidate because project e536c0e1e08340e89b27fd10934f4843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:57.417270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.418199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30d9cb78-4ecf-4231-a9bb-cf0b9d1c18b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.424358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 975/1962] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:57 2022] DELETE /identity/v3/projects/e536c0e1e08340e89b27fd10934f4843 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:57.434065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.434432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.434836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.458829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:57.460185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:57.528776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 988/1963] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:57.537955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.538390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.538758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.559735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:57.561100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:57.569947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 976/1964] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:57.574509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.585046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856208-cb26-4553-bf58-e2e6366b937b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.589676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.590335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.590619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.590895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.591625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:45:57.591929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.593572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.596020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:57.608542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 989/1965] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:45:57.612376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.622118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57072a2a-73eb-4fad-87f9-0b66c2e40f59 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.627301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.627964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.628243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.628555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.629281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.631272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.660087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 977/1966] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:57 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:57.665112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.675079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b953-ec4a-4149-b264-085340672956 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.736824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.737554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.737882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.738163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.738872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.741533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.763702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 990/1967] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:57 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:57.767980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.779572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1aa2e5-008b-48d8-94b8-765447f90f2a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.784632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.785316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.785607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.785906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.786790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.788784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.797359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.799230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:57.802043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 978/1968] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:45:57.806375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.812365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf6d5-6c11-4567-bc1a-c2c69b7791f9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.816419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74956-a0fb-4e9b-9da6-e38050cc43ae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.817419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.818535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.818937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.819170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] PATH_INFO: `/v3/users/7784bf417d614947a8632e5b00805c52` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.820418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7784bf417d614947a8632e5b00805c52)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.821934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.822814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.822931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.823163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.823511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.824332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.826006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.847899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 979/1969] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:45:57.857620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:57.870159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa37a-6b35-469e-a1ae-0700801a55f1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:57.876241 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6d595-3a51-4134-a752-9aed00e37c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:57.877463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d595-3a51-4134-a752-9aed00e37c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.877761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d595-3a51-4134-a752-9aed00e37c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.878048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d595-3a51-4134-a752-9aed00e37c0f admin admin] PATH_INFO: `/v3/projects/f2014a7fc5164316abcf4d17273bf9e5/users/1c59c62cd1f94cd0a2acc604e64ccf77/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.889929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7784bf417d614947a8632e5b00805c52'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.890696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7784bf417d614947a8632e5b00805c52'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.898845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6d595-3a51-4134-a752-9aed00e37c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2014a7fc5164316abcf4d17273bf9e5, user_id=1c59c62cd1f94cd0a2acc604e64ccf77, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:57.899476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7784bf417d614947a8632e5b00805c52'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.900507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7784bf417d614947a8632e5b00805c52'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.901675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6d595-3a51-4134-a752-9aed00e37c0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:57.910226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-748d4407-653c-4165-a80a-2d209bbe4026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7784bf417d614947a8632e5b00805c52'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:57.938823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 980/1970] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:57 2022] PUT /identity/v3/projects/f2014a7fc5164316abcf4d17273bf9e5/users/1c59c62cd1f94cd0a2acc604e64ccf77/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:57.951056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9597f66-9622-4255-af64-052bbd26631f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.951372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9597f66-9622-4255-af64-052bbd26631f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.951637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9597f66-9622-4255-af64-052bbd26631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.958569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 991/1971] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:57 2022] DELETE /identity/v3/users/7784bf417d614947a8632e5b00805c52 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:57.963695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:57.964435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:57.964730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:57.997839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:58.001333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.016555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f9597f66-9622-4255-af64-052bbd26631f None None] MFA Rules not processed for user `1c59c62cd1f94cd0a2acc604e64ccf77`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:58.019114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9597f66-9622-4255-af64-052bbd26631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.131852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.147703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 992/1972] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:58.148684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.150072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.153523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 981/1973] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:45:58.154531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9597f66-9622-4255-af64-052bbd26631f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.156084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9597f66-9622-4255-af64-052bbd26631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.170164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2992ac-7774-41b3-a1d5-6099eb0f66c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.174353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9597f66-9622-4255-af64-052bbd26631f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.175253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.176149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.176461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.176743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] PATH_INFO: `/v3/users/13ef43517ff74c21a5fd4bb95a0c52f2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.178902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ef43517ff74c21a5fd4bb95a0c52f2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.181044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.182729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.183509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.183990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.184282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.185033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.188106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.191984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.230522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 982/1974] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:45:58 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:58.239643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.264987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ef43517ff74c21a5fd4bb95a0c52f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:58.266107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ef43517ff74c21a5fd4bb95a0c52f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:58.279741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ef43517ff74c21a5fd4bb95a0c52f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:58.282128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ef43517ff74c21a5fd4bb95a0c52f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:58.295542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ef43517ff74c21a5fd4bb95a0c52f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:58.323100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 993/1975] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:58 2022] DELETE /identity/v3/users/13ef43517ff74c21a5fd4bb95a0c52f2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:58.324001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.327440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.337811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8effca-7be7-45e7-aa85-3fe4d000990b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.417331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95aab0-eeb2-44cf-9ebb-2582dd565765 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.423816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.424833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.425403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.425689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.429622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.439011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328e69-58bd-47ba-9f6f-d134c91ebcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.442606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328e69-58bd-47ba-9f6f-d134c91ebcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.442900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328e69-58bd-47ba-9f6f-d134c91ebcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.443307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328e69-58bd-47ba-9f6f-d134c91ebcbe admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.444588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03328e69-58bd-47ba-9f6f-d134c91ebcbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.448980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03328e69-58bd-47ba-9f6f-d134c91ebcbe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.495549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 983/1976] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:45:58 2022] POST /identity/v3/users => generated 290 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:45:58.496962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.497258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.497541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.517367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:58.518660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.533035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 984/1977] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:58.534007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.535803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:58.551772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064cd87-5907-4b22-9875-bbeeea0e294b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.552142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.554514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.557951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.559111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.559623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.560216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.561133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.562928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.565611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 994/1978] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:58.566733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.582267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28959b8b-ac0b-4557-abba-ab9ac9619fe5 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.584610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 985/1979] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:58.590600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.590776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.591455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.591831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.592186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.602718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.604156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961beae2-8269-42ca-a151-8b56d1bae0d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.605070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.609480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.612618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.614111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.614545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] PATH_INFO: `/v3/projects/a8ddb94add09497bb5f93d2d17070948/users/1640b2dcee6e471a8dc7494e69e836e7/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.621482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 995/1980] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:58.643285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ddb94add09497bb5f93d2d17070948, user_id=1640b2dcee6e471a8dc7494e69e836e7, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.646319 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.667307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 986/1981] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:58 2022] PUT /identity/v3/projects/a8ddb94add09497bb5f93d2d17070948/users/1640b2dcee6e471a8dc7494e69e836e7/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:45:58.671904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.710985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa81e-b615-4a2c-b052-b0378ae105d8 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.725300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:58.814710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7357ff-26d1-42d7-b72b-935360a9bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.815337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7357ff-26d1-42d7-b72b-935360a9bd7c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.815588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7357ff-26d1-42d7-b72b-935360a9bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.815876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7357ff-26d1-42d7-b72b-935360a9bd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.817089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7357ff-26d1-42d7-b72b-935360a9bd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.819407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7357ff-26d1-42d7-b72b-935360a9bd7c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.836639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 996/1982] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:45:58.838453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:58.850233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c25ee6-ab96-487b-85ac-0be12996c47c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:58.855330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:58.856412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.856762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.857780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] PATH_INFO: `/v3/projects/a8ddb94add09497bb5f93d2d17070948/users/1640b2dcee6e471a8dc7494e69e836e7/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.859964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ddb94add09497bb5f93d2d17070948, user_id=1640b2dcee6e471a8dc7494e69e836e7, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:58.862971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:58.882038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 987/1983] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:45:58 2022] PUT /identity/v3/projects/a8ddb94add09497bb5f93d2d17070948/users/1640b2dcee6e471a8dc7494e69e836e7/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:58.887796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:58.888821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:58.888821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:58.921024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] MFA Rules not processed for user `1640b2dcee6e471a8dc7494e69e836e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:45:58.922233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.016687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 997/1984] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:45:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:45:59.025945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.028578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.074329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:59.158659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1be4f9-c441-4faa-9a87-24255d43b41c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.167852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.168720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.171726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.172066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.186852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.189675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.204731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 988/1985] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:45:59.305367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.307320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.323350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e1495-f873-4056-950b-cf463cc3c65e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.327451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.328152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.328472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.328747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.330993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.399815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.401498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.416652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 998/1986] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:45:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:45:59.611703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.613402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.641305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.643935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.727813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:45:59.763087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6c7a9-8c18-49f0-ab74-7aff8dd42ab1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.785282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.786179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.786460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.790325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] PATH_INFO: `/v3/projects/be3a448c44af40d9b0297a48948f7f38` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.792119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3a448c44af40d9b0297a48948f7f38)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.794473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.812233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49107f7f-4c9b-4d4f-8210-5dff478c2390 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:45:59.822302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:45:59.824437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:45:59.825234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:45:59.825728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] PATH_INFO: `/v3/projects/66580b18e5c54b3f9ecf199e046f3cd2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:45:59.827831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66580b18e5c54b3f9ecf199e046f3cd2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:45:59.834620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:45:59.903306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3a448c44af40d9b0297a48948f7f38', 'request_id': 'req-62350551-df6f-481d-a4de-14aa4acb2560'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.915119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3a448c44af40d9b0297a48948f7f38', 'request_id': 'req-62350551-df6f-481d-a4de-14aa4acb2560'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.928155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3a448c44af40d9b0297a48948f7f38', 'request_id': 'req-62350551-df6f-481d-a4de-14aa4acb2560'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.947287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66580b18e5c54b3f9ecf199e046f3cd2', 'request_id': 'req-1d561615-f434-43db-977f-607aad2f2c91'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.952091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] The token cache is being invalidate because project be3a448c44af40d9b0297a48948f7f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:59.952428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.956684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.959126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 989/1987] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:59 2022] DELETE /identity/v3/projects/be3a448c44af40d9b0297a48948f7f38 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:45:59.963870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:45:59.966634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:45:59.967672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66580b18e5c54b3f9ecf199e046f3cd2', 'request_id': 'req-1d561615-f434-43db-977f-607aad2f2c91'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.982147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66580b18e5c54b3f9ecf199e046f3cd2', 'request_id': 'req-1d561615-f434-43db-977f-607aad2f2c91'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.994795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] The token cache is being invalidate because project 66580b18e5c54b3f9ecf199e046f3cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:45:59.995596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.996057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:45:59.997959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 999/1988] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:45:59 2022] DELETE /identity/v3/projects/66580b18e5c54b3f9ecf199e046f3cd2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:00.002143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.003961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.071568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62350551-df6f-481d-a4de-14aa4acb2560 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.074625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d561615-f434-43db-977f-607aad2f2c91 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.076804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.077684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.078038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.078248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] PATH_INFO: `/v3/users/38b6695ef3bc494ea62a067fc3a438c9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.079570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b6695ef3bc494ea62a067fc3a438c9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.079856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.080911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.081257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.081572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.081676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] PATH_INFO: `/v3/users/578d7de9e7aa43ee9bcc5216418126ee` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.082868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=578d7de9e7aa43ee9bcc5216418126ee)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.083235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:00.085531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.133455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b6695ef3bc494ea62a067fc3a438c9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.134973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578d7de9e7aa43ee9bcc5216418126ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.135669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578d7de9e7aa43ee9bcc5216418126ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.143089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578d7de9e7aa43ee9bcc5216418126ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.143765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578d7de9e7aa43ee9bcc5216418126ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.146599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b6695ef3bc494ea62a067fc3a438c9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.146863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:00.147765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b6695ef3bc494ea62a067fc3a438c9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.153313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578d7de9e7aa43ee9bcc5216418126ee'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.157753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b6695ef3bc494ea62a067fc3a438c9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.158539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b6695ef3bc494ea62a067fc3a438c9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.171828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 990/1989] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:45:59 2022] DELETE /identity/v3/users/38b6695ef3bc494ea62a067fc3a438c9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:00.176541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1000/1990] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:00 2022] DELETE /identity/v3/users/578d7de9e7aa43ee9bcc5216418126ee => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:00.569957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.571531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.573327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.575991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.665951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa75e66-0dde-451e-8807-426adaac13f4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.667544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65390878-65f5-430d-b437-f36fd47a00e6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.670636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.671514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.671789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.672061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] PATH_INFO: `/v3/projects/d643bbf093804b96b3b2a5cad6a240be` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.672491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.673350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.673481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d643bbf093804b96b3b2a5cad6a240be)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.673673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.674043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] PATH_INFO: `/v3/projects/62d6c66d76ec4d3e8b7d41a21ec1e34a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.675263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.675495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d6c66d76ec4d3e8b7d41a21ec1e34a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.677618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.727859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d6c66d76ec4d3e8b7d41a21ec1e34a', 'request_id': 'req-79a5018c-0bba-4852-ac6a-2956119af3ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.728353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd643bbf093804b96b3b2a5cad6a240be', 'request_id': 'req-74ed598e-0610-4b74-af87-8fb17d10b136'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.733800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d6c66d76ec4d3e8b7d41a21ec1e34a', 'request_id': 'req-79a5018c-0bba-4852-ac6a-2956119af3ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.738084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd643bbf093804b96b3b2a5cad6a240be', 'request_id': 'req-74ed598e-0610-4b74-af87-8fb17d10b136'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.741960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d6c66d76ec4d3e8b7d41a21ec1e34a', 'request_id': 'req-79a5018c-0bba-4852-ac6a-2956119af3ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.746368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd643bbf093804b96b3b2a5cad6a240be', 'request_id': 'req-74ed598e-0610-4b74-af87-8fb17d10b136'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.754687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] The token cache is being invalidate because project 62d6c66d76ec4d3e8b7d41a21ec1e34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:00.754981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.755844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-79a5018c-0bba-4852-ac6a-2956119af3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.757597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 991/1991] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:00 2022] DELETE /identity/v3/projects/62d6c66d76ec4d3e8b7d41a21ec1e34a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:00.761013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] The token cache is being invalidate because project d643bbf093804b96b3b2a5cad6a240be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:00.761013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.761013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74ed598e-0610-4b74-af87-8fb17d10b136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:00.761485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1001/1992] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:00 2022] DELETE /identity/v3/projects/d643bbf093804b96b3b2a5cad6a240be => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:00.762401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f847-0e47-45bc-b4a9-cc3735fc9a3a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.762690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f847-0e47-45bc-b4a9-cc3735fc9a3a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.762994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f847-0e47-45bc-b4a9-cc3735fc9a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.766321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681e66d-17ad-452b-bc06-6e4e568ab3f7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.766618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681e66d-17ad-452b-bc06-6e4e568ab3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.766931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681e66d-17ad-452b-bc06-6e4e568ab3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.781951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7fb0f847-0e47-45bc-b4a9-cc3735fc9a3a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:00.783163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fb0f847-0e47-45bc-b4a9-cc3735fc9a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.787667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a681e66d-17ad-452b-bc06-6e4e568ab3f7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:00.788346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:00.788947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a681e66d-17ad-452b-bc06-6e4e568ab3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.860547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1002/1993] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:00.866959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 992/1994] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:00.869329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.869706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.870058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.871866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.873010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.873307 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.886991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:00.895800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:00.897843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.900476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:00.901635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:00.906963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 993/1995] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:00.912557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1003/1996] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:00.917438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.919936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.927834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3485d31-564e-4ce6-9db7-980b05086c81 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.931206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04134539-498f-4dbf-abe9-dd83bb48064d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.933090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.933927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.934377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.934804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.935685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:00.935924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.936392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.936628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.936918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.937204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.937935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:00.938066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.938271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.940147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.952482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 994/1997] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:00.956342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1004/1998] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:00.965700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.965888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:00.968171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2523fa-836c-4054-b100-c75b6a56294f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.973077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.973750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.974031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.974809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.975561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.977074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197df06-5a22-48ce-9918-b89d82379fc6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:00.977564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:00.985095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:00.985095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:00.985095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:00.985095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:00.985455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:00.989962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.023176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 995/1999] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:00 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:01.023431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1005/2000] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:00 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:01.024139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.027604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.119420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cb3d5-b79f-453d-89e4-899700b3d85e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.130915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.131614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.132698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.132982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.133694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.135957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.164638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5671-07a2-4d08-bdad-cfa131da389a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.170722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.171499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.171889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.172167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.172924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.175104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.181931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1006/2001] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:01.194293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.206423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90f48-42f9-41fe-923b-996d95c67701 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.209396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 996/2002] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:01.213390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.216296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.216704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.216992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.217267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.222063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.222773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.234952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a8d22-b23e-4c68-a4c6-12e5dc1c5902 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.237412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1007/2003] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:01.242201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.247514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.249603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.250093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.251070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.252350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.254139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67418914-aa87-45c4-bc18-bbec886ae4d9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.256535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.260797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.261510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.261973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.262230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.262994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.271612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 997/2004] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:01.275178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.275724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.289364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89905f49-97ae-4f01-b6c9-1a2e025c07ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.295774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.296839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.297193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.297631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.298957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.306112 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1008/2005] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:01.306304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.308625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.319877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 998/2006] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:01.323706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.334176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b00269-920e-42f2-b1ce-a336c3d8ae68 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.338716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfbee6-4aa5-4248-ab00-9ad22ade4cd6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.343777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8145-561a-42a3-9fb1-b828e42845ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.344532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8ad59-2307-41b1-b970-c6dbd30a2938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.345037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8145-561a-42a3-9fb1-b828e42845ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.345328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8145-561a-42a3-9fb1-b828e42845ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.345609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8145-561a-42a3-9fb1-b828e42845ba admin admin] PATH_INFO: `/v3/projects/35377f8c2c6a40e5bd85b461a691f085/users/fa2436bd8aec4ef4b9d1744af1ba5967/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.347125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8ad59-2307-41b1-b970-c6dbd30a2938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.347503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8ad59-2307-41b1-b970-c6dbd30a2938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.347979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8ad59-2307-41b1-b970-c6dbd30a2938 admin admin] PATH_INFO: `/v3/projects/ad9979285ec94fa1aee7694c0f85f8aa/users/5bf18be4acdb43a795b930d24c057066/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.374217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f8145-561a-42a3-9fb1-b828e42845ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=35377f8c2c6a40e5bd85b461a691f085, user_id=fa2436bd8aec4ef4b9d1744af1ba5967, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.377712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f8145-561a-42a3-9fb1-b828e42845ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.380966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8ad59-2307-41b1-b970-c6dbd30a2938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9979285ec94fa1aee7694c0f85f8aa, user_id=5bf18be4acdb43a795b930d24c057066, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.384197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8ad59-2307-41b1-b970-c6dbd30a2938 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.397454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1009/2007] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:01 2022] PUT /identity/v3/projects/35377f8c2c6a40e5bd85b461a691f085/users/fa2436bd8aec4ef4b9d1744af1ba5967/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:01.408561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d042ff7-a087-436a-a241-e7383704992e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.408854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d042ff7-a087-436a-a241-e7383704992e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.409184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d042ff7-a087-436a-a241-e7383704992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.413555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 999/2008] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:01 2022] PUT /identity/v3/projects/ad9979285ec94fa1aee7694c0f85f8aa/users/5bf18be4acdb43a795b930d24c057066/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:01.423720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.424010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.424323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.441090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9d042ff7-a087-436a-a241-e7383704992e None None] MFA Rules not processed for user `fa2436bd8aec4ef4b9d1744af1ba5967`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:01.442388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d042ff7-a087-436a-a241-e7383704992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:01.445808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:01.455782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] MFA Rules not processed for user `5bf18be4acdb43a795b930d24c057066`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:01.458269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:01.524390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:01.530759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1010/2009] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:01.533710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1000/2010] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:01.535527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d042ff7-a087-436a-a241-e7383704992e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.539205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.615392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d17b4-070e-41dd-ae74-ad5bf527f13e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.619660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d042ff7-a087-436a-a241-e7383704992e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.623068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.623795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.624085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.624441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.625165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.625948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.627077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.627362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.627641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.628072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.629066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.632628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.679201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1001/2012] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:01.679424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1011/2012] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:01.680962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.681639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.806043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a22e2-0aa0-413a-8194-d133b2e3664c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.810080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73c0b-a3b7-4239-80ae-6b0a82eab04d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.813041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.814362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.814950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.815057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.815518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.815882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.816401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.816712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.817377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.817691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.820638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.824776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.855518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1002/2013] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:01.858794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1012/2014] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:01 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:01.859990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.869510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.873574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934be10c-a0ff-43ce-81aa-3a667648fb95 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.881868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554a871-83c1-4aae-b3cf-a0c8e29201f1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.885591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.887573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.888509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.889473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.889595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.889843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.890189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.891042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.891484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.892615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.892862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.898763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:01.911151 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1013/2015] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:01.915838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.921281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1003/2016] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:01.927166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:01.934987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:01.944761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf69d04-22c0-4f4d-b6a7-072d30c8e206 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.951479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cffa0-302a-404f-a76e-8001eb238f59 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:01.952525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.953463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.953731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.954525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] PATH_INFO: `/v3/projects/ac2246327d6e44d9b2289c0dbcc363e5/users/6f6b8332331a4ca982d460dcdeb73fd2/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.960343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:01.961351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:01.961701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:01.962847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] PATH_INFO: `/v3/projects/3ca772ea0f724d11a691eaf6884506c0/users/8fc982d6f49047e0b2c1a1e889692080/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:01.985546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca772ea0f724d11a691eaf6884506c0, user_id=8fc982d6f49047e0b2c1a1e889692080, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:01.988576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.002266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2246327d6e44d9b2289c0dbcc363e5, user_id=6f6b8332331a4ca982d460dcdeb73fd2, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.007754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.008576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1004/2017] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:01 2022] PUT /identity/v3/projects/3ca772ea0f724d11a691eaf6884506c0/users/8fc982d6f49047e0b2c1a1e889692080/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:02.015788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.033132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1014/2018] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:01 2022] PUT /identity/v3/projects/ac2246327d6e44d9b2289c0dbcc363e5/users/6f6b8332331a4ca982d460dcdeb73fd2/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:02.034954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.095707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:02.106346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013ce3-2a7e-401c-bb39-3b8457a20679 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.129904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463446-a489-4ee1-ab6b-20011ab2fb70 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.136292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.137567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.137763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.138033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.139089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.140879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.153049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1015/2019] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:02.157591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.168355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c857b2-309d-4968-b887-86cf4e7007fb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.173187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24256e-e377-4b71-af59-932d2ffb5fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.174089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24256e-e377-4b71-af59-932d2ffb5fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.174377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24256e-e377-4b71-af59-932d2ffb5fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.174990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24256e-e377-4b71-af59-932d2ffb5fc7 admin admin] PATH_INFO: `/v3/projects/ac2246327d6e44d9b2289c0dbcc363e5/users/6f6b8332331a4ca982d460dcdeb73fd2/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.175102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.175962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.176452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.176703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24256e-e377-4b71-af59-932d2ffb5fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2246327d6e44d9b2289c0dbcc363e5, user_id=6f6b8332331a4ca982d460dcdeb73fd2, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.176818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.178537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.180201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.180441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24256e-e377-4b71-af59-932d2ffb5fc7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.195061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1005/2021] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:02.196132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.200232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1016/2021] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:02 2022] PUT /identity/v3/projects/ac2246327d6e44d9b2289c0dbcc363e5/users/6f6b8332331a4ca982d460dcdeb73fd2/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:02.203698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.203993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.204280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.210391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e84fc4-edf4-4684-8e66-099e47d97d10 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.237059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] MFA Rules not processed for user `6f6b8332331a4ca982d460dcdeb73fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:02.238327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:02.281047 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678d080-1de9-431d-81ff-007e30e8f5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.282230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678d080-1de9-431d-81ff-007e30e8f5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.282650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678d080-1de9-431d-81ff-007e30e8f5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.283125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678d080-1de9-431d-81ff-007e30e8f5e4 admin admin] PATH_INFO: `/v3/projects/3ca772ea0f724d11a691eaf6884506c0/users/8fc982d6f49047e0b2c1a1e889692080/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.285382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678d080-1de9-431d-81ff-007e30e8f5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca772ea0f724d11a691eaf6884506c0, user_id=8fc982d6f49047e0b2c1a1e889692080, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.288289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678d080-1de9-431d-81ff-007e30e8f5e4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.299772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1006/2022] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:02 2022] PUT /identity/v3/projects/3ca772ea0f724d11a691eaf6884506c0/users/8fc982d6f49047e0b2c1a1e889692080/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:02.310392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.310717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.311000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.323924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1017/2023] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:02.335836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.337265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:02.344461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] MFA Rules not processed for user `8fc982d6f49047e0b2c1a1e889692080`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:02.346340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:02.353963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:02.430250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0d62e-6bf5-4fb8-849e-e06ec3ae38ef None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.449557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1007/2024] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:02.460549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.462094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:02.486355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.487023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.487312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.487595 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.503988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:02.523382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666ccab-0de1-4086-82d0-32a6c9c72e33 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.527202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.527861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.528145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.528494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.538463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.539916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.550405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1008/2025] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:02.558391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.559889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.570441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1018/2026] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:02.672152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:02.682390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb5aa4-317b-4fc8-ac72-eae339ef1095 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:02.688797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af2bb5-4b55-4a4f-bafd-4e0ac36a12ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:02.689450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af2bb5-4b55-4a4f-bafd-4e0ac36a12ef service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:02.689739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af2bb5-4b55-4a4f-bafd-4e0ac36a12ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:02.690020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af2bb5-4b55-4a4f-bafd-4e0ac36a12ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:02.753662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af2bb5-4b55-4a4f-bafd-4e0ac36a12ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:02.755250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af2bb5-4b55-4a4f-bafd-4e0ac36a12ef service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:02.764388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1009/2027] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:03.276402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.368560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511830ef-6ea2-4b51-9f2c-728e8647334f service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.373708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.374611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.374892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.375264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] PATH_INFO: `/v3/users/fa2436bd8aec4ef4b9d1744af1ba5967` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.376506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa2436bd8aec4ef4b9d1744af1ba5967)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.379050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.521535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa2436bd8aec4ef4b9d1744af1ba5967'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.522234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa2436bd8aec4ef4b9d1744af1ba5967'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.530020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa2436bd8aec4ef4b9d1744af1ba5967'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.530794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa2436bd8aec4ef4b9d1744af1ba5967'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.540609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa2436bd8aec4ef4b9d1744af1ba5967'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.569605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1019/2028] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:03 2022] DELETE /identity/v3/users/fa2436bd8aec4ef4b9d1744af1ba5967 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:03.571998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.572654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.573002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.573696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.575569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:03.594056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:03.595291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:03.614158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.644129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:03.650564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbc49e-61df-4036-ac5b-82068ec2e9a4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.655520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.656494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.656812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.657116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] PATH_INFO: `/v3/users/1c59c62cd1f94cd0a2acc604e64ccf77` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.658378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c59c62cd1f94cd0a2acc604e64ccf77)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.660408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.663659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1010/2029] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:03.674869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.677729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:03.714844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c59c62cd1f94cd0a2acc604e64ccf77'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.715682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c59c62cd1f94cd0a2acc604e64ccf77'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.722593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c59c62cd1f94cd0a2acc604e64ccf77'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.723382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c59c62cd1f94cd0a2acc604e64ccf77'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.731560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a251d5c-85f3-434d-a8db-dc5a30fe2f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c59c62cd1f94cd0a2acc604e64ccf77'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:03.737296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ff6e-d16f-4c44-8e2d-667b5e0402ed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.741926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.742585 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.742957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.743145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.745008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:03.750242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1020/2030] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:03 2022] DELETE /identity/v3/users/1c59c62cd1f94cd0a2acc604e64ccf77 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:03.754551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.754842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.755127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.760198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.761767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.772419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1011/2031] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:03.773543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:03.774776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:03.838560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1021/2032] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:03.851984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:03.854408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:03.871001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2178b-5930-42f9-a31a-30174e812b68 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:03.944336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:03.945081 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:03.945318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:03.945600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:03.956022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:03.957597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:03.969786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1012/2033] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:04.183921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.185546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:04.198678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1575-7b13-4234-a794-6425b599e641 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.204466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.205462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.205748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.206061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] PATH_INFO: `/v3/projects/35377f8c2c6a40e5bd85b461a691f085` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.207545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35377f8c2c6a40e5bd85b461a691f085)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.211444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.282755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:04.285703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35377f8c2c6a40e5bd85b461a691f085', 'request_id': 'req-bd378e6b-4bcd-4e5f-a015-f353486180a5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.301021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35377f8c2c6a40e5bd85b461a691f085', 'request_id': 'req-bd378e6b-4bcd-4e5f-a015-f353486180a5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.311538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35377f8c2c6a40e5bd85b461a691f085', 'request_id': 'req-bd378e6b-4bcd-4e5f-a015-f353486180a5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.329986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] The token cache is being invalidate because project 35377f8c2c6a40e5bd85b461a691f085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:04.331675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.332826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.335584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1022/2034] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:04 2022] DELETE /identity/v3/projects/35377f8c2c6a40e5bd85b461a691f085 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:04.340472 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.351932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c388730-23d0-498e-af54-8fe0bdb047f7 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.436144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.437075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.437338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.437610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] PATH_INFO: `/v3/users/8fc982d6f49047e0b2c1a1e889692080` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.438837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc982d6f49047e0b2c1a1e889692080)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.440904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.460868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.462774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:04.480852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd378e6b-4bcd-4e5f-a015-f353486180a5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.496290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.497647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:04.498630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.498973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.499234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] PATH_INFO: `/v3/projects/f2014a7fc5164316abcf4d17273bf9e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.501135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2014a7fc5164316abcf4d17273bf9e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.504998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.519236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc982d6f49047e0b2c1a1e889692080'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.535638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc982d6f49047e0b2c1a1e889692080'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.536283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc982d6f49047e0b2c1a1e889692080'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.546665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc982d6f49047e0b2c1a1e889692080'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.547587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc982d6f49047e0b2c1a1e889692080'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.567938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1013/2035] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:04 2022] DELETE /identity/v3/users/8fc982d6f49047e0b2c1a1e889692080 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:04.571693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2014a7fc5164316abcf4d17273bf9e5', 'request_id': 'req-701a0371-e0e0-47c7-8c76-d15ddd471dc3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.584016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2014a7fc5164316abcf4d17273bf9e5', 'request_id': 'req-701a0371-e0e0-47c7-8c76-d15ddd471dc3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.603198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2014a7fc5164316abcf4d17273bf9e5', 'request_id': 'req-701a0371-e0e0-47c7-8c76-d15ddd471dc3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.635518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] The token cache is being invalidate because project f2014a7fc5164316abcf4d17273bf9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:04.635817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.637325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.640582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1023/2036] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:04 2022] DELETE /identity/v3/projects/f2014a7fc5164316abcf4d17273bf9e5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:04.644373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.647501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:04.729721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403faf-9c43-437a-8a23-7bdab09ffcf7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:04.734500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:04.735393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:04.735667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:04.735964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] PATH_INFO: `/v3/users/1640b2dcee6e471a8dc7494e69e836e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:04.737281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1640b2dcee6e471a8dc7494e69e836e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:04.739228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:04.792829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1640b2dcee6e471a8dc7494e69e836e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.811288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1640b2dcee6e471a8dc7494e69e836e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.812426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1640b2dcee6e471a8dc7494e69e836e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.823822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1640b2dcee6e471a8dc7494e69e836e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.824817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1640b2dcee6e471a8dc7494e69e836e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:04.838835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1014/2037] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:04 2022] DELETE /identity/v3/users/1640b2dcee6e471a8dc7494e69e836e7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:04.937515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:04.939085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.041188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a0371-e0e0-47c7-8c76-d15ddd471dc3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.112444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.113374 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.113662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.113948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] PATH_INFO: `/v3/projects/3ca772ea0f724d11a691eaf6884506c0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.115931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca772ea0f724d11a691eaf6884506c0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.118127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.126501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:05.150313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.152219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.170169 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06163-6dff-4b51-a4fe-5a0ee47ca051 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.186683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.187568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.187847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.188123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] PATH_INFO: `/v3/projects/a8ddb94add09497bb5f93d2d17070948` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.189462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ddb94add09497bb5f93d2d17070948)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.192056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.202255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca772ea0f724d11a691eaf6884506c0', 'request_id': 'req-af89cb2a-57c7-48da-bdb4-167380044f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.216268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca772ea0f724d11a691eaf6884506c0', 'request_id': 'req-af89cb2a-57c7-48da-bdb4-167380044f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.225041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca772ea0f724d11a691eaf6884506c0', 'request_id': 'req-af89cb2a-57c7-48da-bdb4-167380044f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.236884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:05.237219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] The token cache is being invalidate because project 3ca772ea0f724d11a691eaf6884506c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:05.237939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.238721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-af89cb2a-57c7-48da-bdb4-167380044f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.241387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1024/2038] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:04 2022] DELETE /identity/v3/projects/3ca772ea0f724d11a691eaf6884506c0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:05.247411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4944474-e82b-4545-ac56-ac26369472b7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.247705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4944474-e82b-4545-ac56-ac26369472b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.250540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4944474-e82b-4545-ac56-ac26369472b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.265474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ddb94add09497bb5f93d2d17070948', 'request_id': 'req-59dde065-7f72-4f3b-af7b-c23c2aed89be'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.269946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c4944474-e82b-4545-ac56-ac26369472b7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:05.271726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4944474-e82b-4545-ac56-ac26369472b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.272344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ddb94add09497bb5f93d2d17070948', 'request_id': 'req-59dde065-7f72-4f3b-af7b-c23c2aed89be'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.280572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ddb94add09497bb5f93d2d17070948', 'request_id': 'req-59dde065-7f72-4f3b-af7b-c23c2aed89be'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.294145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] The token cache is being invalidate because project a8ddb94add09497bb5f93d2d17070948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:05.294525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.295259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-59dde065-7f72-4f3b-af7b-c23c2aed89be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.297009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1015/2039] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:05 2022] DELETE /identity/v3/projects/a8ddb94add09497bb5f93d2d17070948 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:05.301736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea80ac8-f2e8-46d8-bbd7-5248f635531b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.302033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea80ac8-f2e8-46d8-bbd7-5248f635531b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.302369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea80ac8-f2e8-46d8-bbd7-5248f635531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.320740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1ea80ac8-f2e8-46d8-bbd7-5248f635531b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:05.321999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ea80ac8-f2e8-46d8-bbd7-5248f635531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.340135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1025/2040] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:05.347818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.348118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.348465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.367351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:05.368598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.377498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1026/2041] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:05.382176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.391412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d52ac6-1ded-457b-910c-26f1f4048eaa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.391592 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1016/2042] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:05.396840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.397522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.398095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.398208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.398897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:05.399213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.399787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.400089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.400405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.400875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.422288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1027/2043] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:46:05.422705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:05.423477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.423587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.432548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1017/2044] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:05.434147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00068446-c58e-4b6e-9d6a-ac344d809b67 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.436558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.441855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.442523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.443397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.443720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.444511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.446428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.446756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3a2a8-3f55-4566-8055-2526453eff03 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.452180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.453266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.453609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.453910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.454752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:05.455146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.457040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.470863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1018/2045] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:46:05.471939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.473731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.476490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1028/2046] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:05.477202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.486706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f902a-595a-48ea-b36e-2a120456f8b2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.533265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:05.545699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077a18-ec10-486c-bde1-95afe45fce2c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.552848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.553623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.553980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.554334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.555113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.557346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.581927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.582821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.583101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.583479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] PATH_INFO: `/v3/users/5bf18be4acdb43a795b930d24c057066` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.586648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf18be4acdb43a795b930d24c057066)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.588926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.592950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1029/2047] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:05.592950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.647638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:05.663752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf18be4acdb43a795b930d24c057066'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.685183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b531c-3e34-4725-81cf-4d241cb803ec admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.689196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf18be4acdb43a795b930d24c057066'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.693354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf18be4acdb43a795b930d24c057066'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.698658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742356b-8426-41e0-9b0a-262c78ab8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.699470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742356b-8426-41e0-9b0a-262c78ab8caf admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.699939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742356b-8426-41e0-9b0a-262c78ab8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.700271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742356b-8426-41e0-9b0a-262c78ab8caf admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.701110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742356b-8426-41e0-9b0a-262c78ab8caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.701723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf18be4acdb43a795b930d24c057066'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.702573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf18be4acdb43a795b930d24c057066'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:05.703060 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742356b-8426-41e0-9b0a-262c78ab8caf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.717164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1019/2048] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:05 2022] DELETE /identity/v3/users/5bf18be4acdb43a795b930d24c057066 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:05.718196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.719613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.729106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1030/2049] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:05.730413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.730712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.731136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.751009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:05.752529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.796713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4463a45-b5ea-4eea-baea-6ca0829c2bf8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.801393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.802095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.802446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.802724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.803445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.805460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.827807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1031/2050] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:05.827807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.828077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1020/2051] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:05 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:05.830355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.834633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.843888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7acaf3-acb2-4883-b736-1852ffb086f0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.845269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e55c-3162-4130-a442-6c43e2d205c5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.849295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.850263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.850441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.850550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.850813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.851061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.851346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.851551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.851648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.852365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.853199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.854226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.867270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1032/2052] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:05.868206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.869077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1021/2053] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:05.871874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:05.873037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.886392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88397-e166-422a-ab42-b6d227c61364 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.894939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.897516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.897993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.898529 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.899326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.901581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.912658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1022/2054] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:05.913561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.924460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba5857-cb02-40db-aab8-ca5dffb2551c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.930922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.931591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.931877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.932154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.932932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.934530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.948993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1023/2055] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:05.949857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:05.957839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c8068-0179-4a7f-ab25-5b1aa68c5558 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.960840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f1d50-006a-4d9f-b2bf-bd031da96b36 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:05.962124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.962789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.963081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.963366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.968664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa35f7-dd3b-4178-8612-2bb6f2be741b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:05.969561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa35f7-dd3b-4178-8612-2bb6f2be741b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:05.969845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa35f7-dd3b-4178-8612-2bb6f2be741b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:05.970122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa35f7-dd3b-4178-8612-2bb6f2be741b admin admin] PATH_INFO: `/v3/projects/43ce673eb46b486d9f8cc64a452e01a6/users/ff970e1070c340d6b8d99cd59631cf50/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:05.973630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.975202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.981116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:05.986004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa35f7-dd3b-4178-8612-2bb6f2be741b admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ce673eb46b486d9f8cc64a452e01a6, user_id=ff970e1070c340d6b8d99cd59631cf50, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:05.987996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1033/2056] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:05.988837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa35f7-dd3b-4178-8612-2bb6f2be741b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:05.990722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.001856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dd7c8-b19d-4537-b700-eea55ba630f2 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.004650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1024/2057] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:05 2022] PUT /identity/v3/projects/43ce673eb46b486d9f8cc64a452e01a6/users/ff970e1070c340d6b8d99cd59631cf50/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:06.016486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.016906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.017290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.048151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] MFA Rules not processed for user `ff970e1070c340d6b8d99cd59631cf50`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:06.049726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.084103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16c9a1-cfa3-4564-978a-ec296b6ab635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.085381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c9a1-cfa3-4564-978a-ec296b6ab635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.085572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c9a1-cfa3-4564-978a-ec296b6ab635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.085845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c9a1-cfa3-4564-978a-ec296b6ab635 admin admin] PATH_INFO: `/v3/projects/a90492f52df4447f81d8e8521ba115d7/users/5c63a4a7e3de469ab746d352bbaf7271/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.095739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:06.112994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16c9a1-cfa3-4564-978a-ec296b6ab635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a90492f52df4447f81d8e8521ba115d7, user_id=5c63a4a7e3de469ab746d352bbaf7271, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.116882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16c9a1-cfa3-4564-978a-ec296b6ab635 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.132579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1034/2058] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:05 2022] PUT /identity/v3/projects/a90492f52df4447f81d8e8521ba115d7/users/5c63a4a7e3de469ab746d352bbaf7271/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:06.144050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.144463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.145010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.150581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1025/2059] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:06.154737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.166567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a300b3-e51c-4294-b3a6-947caaf76bfb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.174243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] MFA Rules not processed for user `5c63a4a7e3de469ab746d352bbaf7271`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:06.175527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.245197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.245901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.246187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.246468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.247163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.249123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.266606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1035/2060] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:06.271284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.282895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66631b9-cd1b-4185-a101-11fca21892ec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.284003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1026/2061] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:06.288716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.362476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.364828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.365226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.365450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.366279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.369708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abb963-a69e-4403-b3a6-1a73a2c6535d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.372722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.374919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.375601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.375906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.376184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.376940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.379684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.398548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:06.406882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1027/2062] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:06.407898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1036/2063] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:06.412882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.413036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.415143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.431497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c1f5-0853-43cd-91d6-ad70c0f8b747 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.479089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade72d7-1e5e-40c3-aa6a-a2eb7a6c2f71 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.483554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:06.483980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.484695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.485107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.485286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.485975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.487598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.501329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1037/2064] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:06.501329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.502441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.503350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.503633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.503916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] PATH_INFO: `/v3/projects/ad9979285ec94fa1aee7694c0f85f8aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.505705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9979285ec94fa1aee7694c0f85f8aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.508152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.512530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad734b-d9de-4390-ae2d-2728267006b9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.517349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.518048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.518329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.518610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.519300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.521876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.543410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1038/2065] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/users => generated 299 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:06.544338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.554025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b3655-f758-452f-bb15-10e7e1943725 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.560088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9979285ec94fa1aee7694c0f85f8aa', 'request_id': 'req-b489683e-f491-45d7-b34f-3a3398e7ddaa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.566211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9979285ec94fa1aee7694c0f85f8aa', 'request_id': 'req-b489683e-f491-45d7-b34f-3a3398e7ddaa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.575390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9979285ec94fa1aee7694c0f85f8aa', 'request_id': 'req-b489683e-f491-45d7-b34f-3a3398e7ddaa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.589418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] The token cache is being invalidate because project ad9979285ec94fa1aee7694c0f85f8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:06.589643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.590292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.591878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1028/2066] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:06 2022] DELETE /identity/v3/projects/ad9979285ec94fa1aee7694c0f85f8aa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:06.592698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.594040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.614028 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.615233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.615518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.615797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.616546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.618149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.632588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1039/2067] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:06.632588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.632588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.644709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045fbcd-0f78-4290-9bd8-12867624f042 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.649967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.651064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.651353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.651634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] PATH_INFO: `/v3/users/6f6b8332331a4ca982d460dcdeb73fd2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.652873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6b8332331a4ca982d460dcdeb73fd2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.654808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.665404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489683e-f491-45d7-b34f-3a3398e7ddaa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.670128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.670797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.671083 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.671390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.672114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.673754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.687367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1029/2068] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:06.687367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.687367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.709354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6b8332331a4ca982d460dcdeb73fd2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.710371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a270107-9c67-4021-ba66-51122ebf33b8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.710884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6b8332331a4ca982d460dcdeb73fd2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.715392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.716332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.716626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.716904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] PATH_INFO: `/v3/projects/426e1c49367541c9b3d1000e5b750e76/users/8195de71cf65447cab14607aaff993ae/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.729280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6b8332331a4ca982d460dcdeb73fd2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.731875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6b8332331a4ca982d460dcdeb73fd2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.734150 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426e1c49367541c9b3d1000e5b750e76, user_id=8195de71cf65447cab14607aaff993ae, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:06.736953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:06.747457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8c61e7c-3ef1-4560-bd9f-78b9aaa4046b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6b8332331a4ca982d460dcdeb73fd2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:06.759154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:06.770504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1030/2069] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:06 2022] PUT /identity/v3/projects/426e1c49367541c9b3d1000e5b750e76/users/8195de71cf65447cab14607aaff993ae/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:06.773001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.774838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.796938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1040/2070] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:06 2022] DELETE /identity/v3/users/6f6b8332331a4ca982d460dcdeb73fd2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:06.796938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.796938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.796938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:06.827665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] MFA Rules not processed for user `8195de71cf65447cab14607aaff993ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:06.828940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.854803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:06.873014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ebfe7-0b28-4ed7-abf9-6e1f0a951a36 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:06.924364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1041/2071] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:06.929702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:06.931108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:06.985553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:06.986758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:06.987114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:06.987464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] PATH_INFO: `/v3/projects/e7c90b0d994a46a984997cb3707b4707/users/eab29475337c40c99e8922b3f1af33e7/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.029694 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c90b0d994a46a984997cb3707b4707, user_id=eab29475337c40c99e8922b3f1af33e7, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.040329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.084283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1031/2072] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:06 2022] PUT /identity/v3/projects/e7c90b0d994a46a984997cb3707b4707/users/eab29475337c40c99e8922b3f1af33e7/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:07.088087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.184796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1ae52-6125-43b8-a09a-2f47b10e6968 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.186930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ada628-b1bf-4cec-9af5-fd0aff549482 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.194652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.195834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.196122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.196456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.197640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.199430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.200127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.200344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.200477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.200702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.201409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.203346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.215857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1032/2073] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:07.220878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.234525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18928a-692e-4df2-bb7e-f990b9980093 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.241457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958deffc-073e-4b16-a685-6b606950ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.242519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958deffc-073e-4b16-a685-6b606950ef1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.243340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958deffc-073e-4b16-a685-6b606950ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.243340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958deffc-073e-4b16-a685-6b606950ef1d admin admin] PATH_INFO: `/v3/projects/e7c90b0d994a46a984997cb3707b4707/users/eab29475337c40c99e8922b3f1af33e7/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.245848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958deffc-073e-4b16-a685-6b606950ef1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c90b0d994a46a984997cb3707b4707, user_id=eab29475337c40c99e8922b3f1af33e7, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.249520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958deffc-073e-4b16-a685-6b606950ef1d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.278681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1033/2074] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:07 2022] PUT /identity/v3/projects/e7c90b0d994a46a984997cb3707b4707/users/eab29475337c40c99e8922b3f1af33e7/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:07.289018 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.289403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1042/2075] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:06 2022] POST /identity/v3/projects => generated 377 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:07.289513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.289513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.289647 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.324368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] MFA Rules not processed for user `eab29475337c40c99e8922b3f1af33e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:07.325965 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.354524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85646b-86d9-4e62-989e-3c0c0819630e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.369507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.371412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.371855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.372282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.373233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.387681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.400257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:07.415226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:07.428192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1043/2076] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:07 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:07.430262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.433042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.442219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1034/2077] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:07.466728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312506ca-e1a4-47a9-808f-f5d43e15e97b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.472159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.473298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.473503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.474316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.475253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] PATH_INFO: `/v3/projects/ac2246327d6e44d9b2289c0dbcc363e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.476774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2246327d6e44d9b2289c0dbcc363e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.479234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.487108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461d96-ee36-4369-bc17-1b445125e837 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.493366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.494485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.495027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.495512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.497050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.499793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.512430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1035/2078] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:07.513903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.515743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.555924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2246327d6e44d9b2289c0dbcc363e5', 'request_id': 'req-e536689d-8cb1-4076-bf2c-155705e0da21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.566890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2246327d6e44d9b2289c0dbcc363e5', 'request_id': 'req-e536689d-8cb1-4076-bf2c-155705e0da21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.593685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2246327d6e44d9b2289c0dbcc363e5', 'request_id': 'req-e536689d-8cb1-4076-bf2c-155705e0da21'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.602193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1453250-dac0-42a9-b07c-f1062e48b2a3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.608207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] The token cache is being invalidate because project ac2246327d6e44d9b2289c0dbcc363e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:07.608487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.609510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.611450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1044/2079] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:07 2022] DELETE /identity/v3/projects/ac2246327d6e44d9b2289c0dbcc363e5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:07.612229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.613701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.665048 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.665647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.665934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.666208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.668010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.702934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536689d-8cb1-4076-bf2c-155705e0da21 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.707704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e284f-8e0b-41f1-9c4d-496290efbde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.708755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e284f-8e0b-41f1-9c4d-496290efbde5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.709204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e284f-8e0b-41f1-9c4d-496290efbde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.709478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e284f-8e0b-41f1-9c4d-496290efbde5 admin admin] PATH_INFO: `/v3/projects/6dba70de72e24d1084bec5a8c2042a7f/users/8c6bb70f2ac34ca69d0840889e974fef/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.726276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e284f-8e0b-41f1-9c4d-496290efbde5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dba70de72e24d1084bec5a8c2042a7f, user_id=8c6bb70f2ac34ca69d0840889e974fef, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.728991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e284f-8e0b-41f1-9c4d-496290efbde5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.739846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.741524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.746504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1045/2080] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:07 2022] PUT /identity/v3/projects/6dba70de72e24d1084bec5a8c2042a7f/users/8c6bb70f2ac34ca69d0840889e974fef/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:07.750822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.751121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.751406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.753980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1036/2081] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:07.754787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.772423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:07.773868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.790499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2a52f-509b-404a-b725-315b359172e1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.846763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:07.874896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1046/2082] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:07.875734 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.877162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.892759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825ef6a-8adc-4375-8ecf-c100619ea6b7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.901275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.901980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:07.902457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.902457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.902701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] PATH_INFO: `/v3/users/5c63a4a7e3de469ab746d352bbaf7271` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.904114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c63a4a7e3de469ab746d352bbaf7271)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.905978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.924572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f477be7-c347-47ba-8c40-849ca5f302d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.925335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f477be7-c347-47ba-8c40-849ca5f302d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.925628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f477be7-c347-47ba-8c40-849ca5f302d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.925911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f477be7-c347-47ba-8c40-849ca5f302d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.926785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f477be7-c347-47ba-8c40-849ca5f302d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:07.928884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f477be7-c347-47ba-8c40-849ca5f302d2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:07.939462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1037/2083] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:07.941096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:07.941471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:07.942322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:07.967080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:07.969139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:07.979121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1038/2084] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:07.980422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:07.982014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c63a4a7e3de469ab746d352bbaf7271'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.982954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c63a4a7e3de469ab746d352bbaf7271'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.992076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c63a4a7e3de469ab746d352bbaf7271'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.992564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a907-a496-438d-8896-dcc8d2f3b45e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:07.992990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c63a4a7e3de469ab746d352bbaf7271'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:07.998173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:07.999513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.000000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.000429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] PATH_INFO: `/v3/projects/6dba70de72e24d1084bec5a8c2042a7f/users/8c6bb70f2ac34ca69d0840889e974fef/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.003911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dba70de72e24d1084bec5a8c2042a7f, user_id=8c6bb70f2ac34ca69d0840889e974fef, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.007800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.015216 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-014bebda-12f9-4dc0-a8ac-b3d406c71831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c63a4a7e3de469ab746d352bbaf7271'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:08.021229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1039/2085] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:07 2022] PUT /identity/v3/projects/6dba70de72e24d1084bec5a8c2042a7f/users/8c6bb70f2ac34ca69d0840889e974fef/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:08.022783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.024594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.057422 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1047/2086] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:07 2022] DELETE /identity/v3/users/5c63a4a7e3de469ab746d352bbaf7271 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:08.059790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c0e40-dcc2-43ad-8664-22f038577f60 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.060288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c0e40-dcc2-43ad-8664-22f038577f60 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.060811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c0e40-dcc2-43ad-8664-22f038577f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.098930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8d6c0e40-dcc2-43ad-8664-22f038577f60 None None] MFA Rules not processed for user `8c6bb70f2ac34ca69d0840889e974fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:08.100751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d6c0e40-dcc2-43ad-8664-22f038577f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.127209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b937f98-c291-499d-ac68-97c221a0e169 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.207049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1048/2087] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:08.208584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.208814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.209090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.226243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.226916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.227207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.227485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.228246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:08.228597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.230215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.235458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:08.237450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.242773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1040/2088] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:08.243681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.245362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.250010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1049/2089] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:08.251107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.263028 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda6634-9d7c-4735-814f-967e406d8093 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.267698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.268396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.268731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.269010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.269700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.271949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.304434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1050/2090] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:08 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:08.304434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.305873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.311922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0236-6526-46b9-aebd-f5a022934015 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.327907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:08.337927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:08.372419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1107f-e637-4779-a469-841b12bb0e52 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.375625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.376386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.376507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.376873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.377037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.377284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.377379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.378505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.379316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.445653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.448058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.453462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.455087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.459186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1041/2091] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:08.460168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.469402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1051/2092] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:08.479513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccced7-3700-4568-bd76-a3250da2d625 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.484366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.485044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.485326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.485603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.486294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.488247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.512173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1042/2093] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:08 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:08.516017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.527672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36997624-aac7-47f6-95f2-368b6d8dabb7 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.537711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.538598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.538975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.539284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.540102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.541971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.579109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1052/2094] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:08.585664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.603123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637f2fb-37c5-46b2-b8f4-3e49b6f3e78c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.609850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50a6db-d042-4616-9ad4-dd3ee9146b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.610491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a6db-d042-4616-9ad4-dd3ee9146b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.610777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a6db-d042-4616-9ad4-dd3ee9146b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.611060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a6db-d042-4616-9ad4-dd3ee9146b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.612574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50a6db-d042-4616-9ad4-dd3ee9146b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.614125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50a6db-d042-4616-9ad4-dd3ee9146b60 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.635154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1043/2095] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:08.636873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:08.654484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eaa747-c326-4893-9d34-2e90e76e4ef0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:08.663881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:08.665925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.667089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.667604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] PATH_INFO: `/v3/projects/45fc4215c0114979b662d35507710aae/users/639f2adb8ef546dcaed1f0662fb4020d/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.721146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fc4215c0114979b662d35507710aae, user_id=639f2adb8ef546dcaed1f0662fb4020d, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:08.726014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:08.753585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1053/2096] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:08 2022] PUT /identity/v3/projects/45fc4215c0114979b662d35507710aae/users/639f2adb8ef546dcaed1f0662fb4020d/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:08.766220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:08.766796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:08.767373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:08.813316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] MFA Rules not processed for user `639f2adb8ef546dcaed1f0662fb4020d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:08.814639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:08.940645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1044/2097] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:08.943141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.037476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d0f68-f43f-4981-8ac7-cbc3f2f32ffe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.044118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.046063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.075845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65914f8-5153-4b88-9ba2-aa513aa42719 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.082867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.085757 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.086379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.087110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] PATH_INFO: `/v3/projects/a90492f52df4447f81d8e8521ba115d7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.091827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a90492f52df4447f81d8e8521ba115d7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.093432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.094291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.094712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.094889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.095324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.096908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.102174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.144393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1054/2098] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:08 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:09.144844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:09.147339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.171429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a90492f52df4447f81d8e8521ba115d7', 'request_id': 'req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.184333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:09.184460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a90492f52df4447f81d8e8521ba115d7', 'request_id': 'req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.204414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a90492f52df4447f81d8e8521ba115d7', 'request_id': 'req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.226422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] The token cache is being invalidate because project a90492f52df4447f81d8e8521ba115d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:09.227222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.227663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.229862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1045/2099] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:09 2022] DELETE /identity/v3/projects/a90492f52df4447f81d8e8521ba115d7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:09.233935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.235493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.273880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d796a-c615-4554-8934-c6dba7065655 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.365206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf130-c5d7-4e63-8bfa-062ee0e29ad6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.370346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.371248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.371535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.371924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] PATH_INFO: `/v3/users/eab29475337c40c99e8922b3f1af33e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.373338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab29475337c40c99e8922b3f1af33e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.375403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.385856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.386591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.386945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.387298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.388175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.390243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.421266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1055/2100] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:09 2022] POST /identity/v3/users => generated 296 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:09.425506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.430733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.459121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf556c-ece1-478b-aeb4-d950b7e11be4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.472048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.473191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.473389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab29475337c40c99e8922b3f1af33e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.473619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.474447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.475198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.477103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.489014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1056/2101] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:09.497739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.508492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab29475337c40c99e8922b3f1af33e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.509263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab29475337c40c99e8922b3f1af33e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.511329 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634f5a-d83d-49ca-b001-9dc0b7ccd053 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.518323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.518783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab29475337c40c99e8922b3f1af33e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.519278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.519531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab29475337c40c99e8922b3f1af33e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:09.519635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.519820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] PATH_INFO: `/v3/projects/dc9b495c22e64a2998c7f0b92bd8845e/users/7f14daab604c47299ff3f3ea82c3fe14/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.534885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1046/2102] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:09 2022] DELETE /identity/v3/users/eab29475337c40c99e8922b3f1af33e7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:09.541745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9b495c22e64a2998c7f0b92bd8845e, user_id=7f14daab604c47299ff3f3ea82c3fe14, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.545682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.561371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1057/2103] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:09 2022] PUT /identity/v3/projects/dc9b495c22e64a2998c7f0b92bd8845e/users/7f14daab604c47299ff3f3ea82c3fe14/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:09.565115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.566531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.624809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:09.685932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5962f-69ed-4be6-9032-7ddc97573a1b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.761629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee91add-32e0-4afe-8acf-e9d2de720f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.762311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91add-32e0-4afe-8acf-e9d2de720f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.762603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91add-32e0-4afe-8acf-e9d2de720f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.762881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91add-32e0-4afe-8acf-e9d2de720f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.763607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee91add-32e0-4afe-8acf-e9d2de720f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.765277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee91add-32e0-4afe-8acf-e9d2de720f14 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.775884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1047/2104] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:09.780023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.790757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a2982-0734-4236-870c-2ad10f40bfa8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.796111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.797047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.797334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.797613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] PATH_INFO: `/v3/projects/dc9b495c22e64a2998c7f0b92bd8845e/users/7f14daab604c47299ff3f3ea82c3fe14/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.799583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9b495c22e64a2998c7f0b92bd8845e, user_id=7f14daab604c47299ff3f3ea82c3fe14, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.802871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.813995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1058/2105] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:09 2022] PUT /identity/v3/projects/dc9b495c22e64a2998c7f0b92bd8845e/users/7f14daab604c47299ff3f3ea82c3fe14/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:09.825511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.825929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.826323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.848352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.849335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.856510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] MFA Rules not processed for user `7f14daab604c47299ff3f3ea82c3fe14`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:09.858029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.928324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a0aff-6791-4809-a2be-398a3e836392 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:09.933713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:09.934652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:09.934943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:09.935227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] PATH_INFO: `/v3/projects/e7c90b0d994a46a984997cb3707b4707` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:09.936474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c90b0d994a46a984997cb3707b4707)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:09.938397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:09.942077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1048/2106] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:09.956748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:09.957640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:09.958836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:09.999651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c90b0d994a46a984997cb3707b4707', 'request_id': 'req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:10.009861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c90b0d994a46a984997cb3707b4707', 'request_id': 'req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:10.017044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c90b0d994a46a984997cb3707b4707', 'request_id': 'req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:10.034419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070f066-832b-4942-a192-f433ffc0cc34 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.034598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] The token cache is being invalidate because project e7c90b0d994a46a984997cb3707b4707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:10.034780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:10.035517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-46c5bad4-7b15-4412-b5f9-efdd9ebeb734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:10.037966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1059/2107] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:09 2022] DELETE /identity/v3/projects/e7c90b0d994a46a984997cb3707b4707 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:10.042489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.042785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.043065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.058993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:10.063324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:10.064945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.103132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.103799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.104089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.104417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.110875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.154478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1060/2108] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:10.155385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.157423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.172973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d9112-1d45-42d0-9efe-7e8a9e25ae96 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.178087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.178800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.179167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.179395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.179506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.180904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed91b5-bb0a-42c7-bf8c-0ca4a018a550 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.182093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.193947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1049/2109] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:10.195881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.196377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.196666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.218105 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:10.219739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.243557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1050/2110] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:10.247721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.266254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d60799-df85-4b3f-8c17-efb0746e4663 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.271442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.272494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.272787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.273073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.274730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:10.275057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.276736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.279558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.282530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.296615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1051/2111] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:46:10.296816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1061/2112] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:10.298873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.310245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f914-9266-4371-b4c8-d8ca6b965f61 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.318858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.320332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.334510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.337468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.337759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.338038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.340046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.342401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e90e47-11ae-43a1-af63-0a651382ea7d service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.343014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.347612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.348353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.349548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.349949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.351927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.384920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1052/2113] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:10 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:10.388722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.402063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:10.443950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.445617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.459092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1062/2114] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:10.493788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de521-be84-43df-9fd6-2aafcc9bc489 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.500933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.501977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.502341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.502849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.503892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.511397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.554200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1053/2115] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:10 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:10.555593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.570271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea027c58-6a3e-4fcc-bd9a-2f9cb7f5bc27 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.584723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.584723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.584723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.584723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.585396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.593228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.622116 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1063/2116] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:10.627082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.644187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926e13c-4006-489e-8d97-75cc452a1f47 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.656019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902c6e6-1830-4fc7-887b-de7285ace37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.656731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902c6e6-1830-4fc7-887b-de7285ace37b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.657629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902c6e6-1830-4fc7-887b-de7285ace37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.657629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902c6e6-1830-4fc7-887b-de7285ace37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.658140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902c6e6-1830-4fc7-887b-de7285ace37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.664271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902c6e6-1830-4fc7-887b-de7285ace37b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.695339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1054/2117] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:10.699657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.716909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a768-22f5-46bb-9685-2ed82978d7db admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:10.729423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:10.730475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.730665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.730900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] PATH_INFO: `/v3/projects/4662e14c3a954837ac9ecee366a302b5/users/d62624875a03415e9bc3402b4dc36645/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.767161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4662e14c3a954837ac9ecee366a302b5, user_id=d62624875a03415e9bc3402b4dc36645, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:10.771409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:10.802631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1064/2118] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:10 2022] PUT /identity/v3/projects/4662e14c3a954837ac9ecee366a302b5/users/d62624875a03415e9bc3402b4dc36645/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:10.813289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:10.813652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:10.814008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:10.828886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:10.855586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] MFA Rules not processed for user `d62624875a03415e9bc3402b4dc36645`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:10.857240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.910683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:10.913078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:10.953612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1055/2119] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:10.958375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.004870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e2aae-9cfa-49bc-a497-8953f5fea2fa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.014273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.015312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.015693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.016033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] PATH_INFO: `/v3/users/639f2adb8ef546dcaed1f0662fb4020d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.017376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639f2adb8ef546dcaed1f0662fb4020d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.019389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.053664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d227a0-d927-416f-8c8e-815be5e490f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.059176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.060037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.060371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.060997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.061583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.063851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.073373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639f2adb8ef546dcaed1f0662fb4020d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:11.074145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639f2adb8ef546dcaed1f0662fb4020d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:11.081620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639f2adb8ef546dcaed1f0662fb4020d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:11.082454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639f2adb8ef546dcaed1f0662fb4020d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:11.090932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b9511287-109f-494e-bdd1-c429371a7390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639f2adb8ef546dcaed1f0662fb4020d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:11.095024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1056/2120] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:10 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:11.100762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.102453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.115220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1065/2121] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:10 2022] DELETE /identity/v3/users/639f2adb8ef546dcaed1f0662fb4020d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:11.119785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.120161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.120609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.139480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:11.141023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.166864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:11.183016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaed6b-6bea-4d02-a0aa-e0f32a3ab66d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.213425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:11.214796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1066/2122] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:11.219059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.220430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.248108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.248842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.249157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.249442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.250128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.252070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.276396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1057/2123] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:11 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:11.276396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.276888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.279123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492d224-41c6-47c6-859e-42e872f0e686 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.282945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.283589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.283876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.284155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.285971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.336244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714111b-fd79-4dd1-9332-1b5c3e61c672 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.340457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.341118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.341435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.341718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.351691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.353304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.359598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.361142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.368392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1058/2124] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:11.370665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.372577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1067/2125] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:11.383346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd300d3a-590d-49e2-81d5-30e52f6f1ab9 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.388701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.389374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.389661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.389953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.390680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.392802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.407502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1059/2126] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:11.409097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.421772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f971c-f896-42e2-9b86-bff0720822b1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.426436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.427914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.428444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.428772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] PATH_INFO: `/v3/projects/3ef2567f3b4143bba7463a9e966150fe/users/6dcd8b1d366e40d08a25a2ad42c8ac77/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.446233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef2567f3b4143bba7463a9e966150fe, user_id=6dcd8b1d366e40d08a25a2ad42c8ac77, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.457507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.503611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1068/2127] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:11 2022] PUT /identity/v3/projects/3ef2567f3b4143bba7463a9e966150fe/users/6dcd8b1d366e40d08a25a2ad42c8ac77/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:11.506972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.526389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb96926-c245-40b5-a318-a42269122dd9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.606393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748912c9-001b-47dd-9303-9895d6ee0ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.607111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748912c9-001b-47dd-9303-9895d6ee0ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.607413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748912c9-001b-47dd-9303-9895d6ee0ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.607710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748912c9-001b-47dd-9303-9895d6ee0ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.608522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748912c9-001b-47dd-9303-9895d6ee0ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.610379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748912c9-001b-47dd-9303-9895d6ee0ca8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.621548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1060/2128] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:11.632591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.648827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c2f79-8e1a-40dc-9d29-469c9ae7cc79 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.655106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.656809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.657286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.657714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] PATH_INFO: `/v3/projects/3ef2567f3b4143bba7463a9e966150fe/users/6dcd8b1d366e40d08a25a2ad42c8ac77/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.660674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef2567f3b4143bba7463a9e966150fe, user_id=6dcd8b1d366e40d08a25a2ad42c8ac77, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.663796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.675884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1069/2129] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:11 2022] PUT /identity/v3/projects/3ef2567f3b4143bba7463a9e966150fe/users/6dcd8b1d366e40d08a25a2ad42c8ac77/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:11.691009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.691009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.691009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.723259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] MFA Rules not processed for user `6dcd8b1d366e40d08a25a2ad42c8ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:11.724539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.767884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:11.813464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1061/2130] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:11.826479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.850359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:11.853000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:11.882828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:11.906044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e21cf-f007-4b61-b620-7a7fac8f26aa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.913645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.914720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.915172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.915613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.931299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.937456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:11.952153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1070/2131] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:11.964186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c1595-01cb-4120-9e7a-6a532e490123 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:11.970214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:11.971212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:11.971640 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:11.972144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] PATH_INFO: `/v3/projects/45fc4215c0114979b662d35507710aae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:11.973568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fc4215c0114979b662d35507710aae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:11.976172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.057893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fc4215c0114979b662d35507710aae', 'request_id': 'req-92709723-a2a3-4b28-af6f-45254fe5882b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.068067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fc4215c0114979b662d35507710aae', 'request_id': 'req-92709723-a2a3-4b28-af6f-45254fe5882b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.092377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fc4215c0114979b662d35507710aae', 'request_id': 'req-92709723-a2a3-4b28-af6f-45254fe5882b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.109330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] The token cache is being invalidate because project 45fc4215c0114979b662d35507710aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:12.109940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.111026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.117860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1062/2132] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:11 2022] DELETE /identity/v3/projects/45fc4215c0114979b662d35507710aae => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:12.118244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:12.137052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c908d3-dd60-487d-9ab8-6ffe1c80071c service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:12.214914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:12.215808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.216094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.216414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] PATH_INFO: `/v3/users/7f14daab604c47299ff3f3ea82c3fe14` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.217652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f14daab604c47299ff3f3ea82c3fe14)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:12.219590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.280401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f14daab604c47299ff3f3ea82c3fe14'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.281236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f14daab604c47299ff3f3ea82c3fe14'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.294385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f14daab604c47299ff3f3ea82c3fe14'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.295026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f14daab604c47299ff3f3ea82c3fe14'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.306670 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67abc427-e932-486a-8f63-9ced63c1e2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f14daab604c47299ff3f3ea82c3fe14'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.328417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1071/2133] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:12 2022] DELETE /identity/v3/users/7f14daab604c47299ff3f3ea82c3fe14 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:12.682036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:12.685672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:12.717355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92709723-a2a3-4b28-af6f-45254fe5882b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:12.845994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:12.847085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.847407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.847724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] PATH_INFO: `/v3/users/d62624875a03415e9bc3402b4dc36645` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:12.849234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62624875a03415e9bc3402b4dc36645)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:12.851562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:12.854617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:12.943582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62624875a03415e9bc3402b4dc36645'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.960085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62624875a03415e9bc3402b4dc36645'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.960870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62624875a03415e9bc3402b4dc36645'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.970895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62624875a03415e9bc3402b4dc36645'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.971729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62624875a03415e9bc3402b4dc36645'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:12.992039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1063/2134] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:12 2022] DELETE /identity/v3/users/d62624875a03415e9bc3402b4dc36645 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:12.997175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:12.997602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:12.997776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:13.081013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:13.082684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:13.176487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:13.198010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1072/2135] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:13.233726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:13.235331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:13.305763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3f5c1-a13b-40a6-84ac-d12950520850 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:13.309664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:13.310330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:13.311805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:13.311805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:13.322856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:13.324780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:13.338961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1064/2136] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:13.735680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:13.737337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:13.754293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7fb73-d813-4be6-bff6-7e7a003378d5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:13.761562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:13.762782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:13.763069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:13.763346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] PATH_INFO: `/v3/projects/4662e14c3a954837ac9ecee366a302b5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:13.764744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4662e14c3a954837ac9ecee366a302b5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:13.766818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:13.828818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4662e14c3a954837ac9ecee366a302b5', 'request_id': 'req-178195f7-77af-429d-a707-6c4c47003b2d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:13.836956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4662e14c3a954837ac9ecee366a302b5', 'request_id': 'req-178195f7-77af-429d-a707-6c4c47003b2d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:13.843889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4662e14c3a954837ac9ecee366a302b5', 'request_id': 'req-178195f7-77af-429d-a707-6c4c47003b2d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:13.866800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] The token cache is being invalidate because project 4662e14c3a954837ac9ecee366a302b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:13.867108 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:13.867843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:13.869646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1073/2137] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:13 2022] DELETE /identity/v3/projects/4662e14c3a954837ac9ecee366a302b5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:13.874277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:13.884806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc8a74-a163-4e4e-bcea-ef975a6c265e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:13.888534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:13.891253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:13.978422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:13.979371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:13.980085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:13.980415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] PATH_INFO: `/v3/users/6dcd8b1d366e40d08a25a2ad42c8ac77` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:13.981812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcd8b1d366e40d08a25a2ad42c8ac77)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:13.986254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.003885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178195f7-77af-429d-a707-6c4c47003b2d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.012393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.013230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.013929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.014716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] PATH_INFO: `/v3/projects/dc9b495c22e64a2998c7f0b92bd8845e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.016585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9b495c22e64a2998c7f0b92bd8845e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.018838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.020747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:14.052066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcd8b1d366e40d08a25a2ad42c8ac77'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.076110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcd8b1d366e40d08a25a2ad42c8ac77'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.077138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcd8b1d366e40d08a25a2ad42c8ac77'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.092782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcd8b1d366e40d08a25a2ad42c8ac77'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.093304 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9b495c22e64a2998c7f0b92bd8845e', 'request_id': 'req-9d049939-119f-4ad6-a72a-0130a90b1c8f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.094172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7aec96f8-c259-4424-8edc-3b5c361de9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcd8b1d366e40d08a25a2ad42c8ac77'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.107124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9b495c22e64a2998c7f0b92bd8845e', 'request_id': 'req-9d049939-119f-4ad6-a72a-0130a90b1c8f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.117265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9b495c22e64a2998c7f0b92bd8845e', 'request_id': 'req-9d049939-119f-4ad6-a72a-0130a90b1c8f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.118908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1065/2138] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:13 2022] DELETE /identity/v3/users/6dcd8b1d366e40d08a25a2ad42c8ac77 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:14.169853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] The token cache is being invalidate because project dc9b495c22e64a2998c7f0b92bd8845e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:14.170250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.172922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d049939-119f-4ad6-a72a-0130a90b1c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.176363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1074/2139] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:13 2022] DELETE /identity/v3/projects/dc9b495c22e64a2998c7f0b92bd8845e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:14.182597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.182958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.183246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.206237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:14.208760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:14.287867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1066/2140] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:14.297321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18666319-4649-4fee-851b-e993f87414ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.297972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18666319-4649-4fee-851b-e993f87414ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.299691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18666319-4649-4fee-851b-e993f87414ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.370044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-18666319-4649-4fee-851b-e993f87414ff None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:14.372193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18666319-4649-4fee-851b-e993f87414ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:14.385268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1075/2141] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:14.390184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.406003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cbb39-3424-43f4-9f7d-f1b3be4c65d5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.418452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.419371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.420891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.421276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.422249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:14.422788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.425323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.448080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1067/2142] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:14.450186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18666319-4649-4fee-851b-e993f87414ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.466009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18666319-4649-4fee-851b-e993f87414ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.472612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.473292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.474826 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.475200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.476055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.478654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.502454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:14.550867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1076/2143] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:14 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:14.555657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.571222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d21e8-fb00-4467-8e44-c148d28eb0d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.685833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.686962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.687545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.688087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.689825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.692119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.738973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1068/2144] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:14 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:14.743378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.757463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91154fe-2685-46bc-8c30-b496bc4fecdb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.762095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.764953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:14.779546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.781622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.782963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.783352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.797701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.799361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.810143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f724b-42f6-4ab1-80b6-1696f63d28e5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.822111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1077/2145] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:14.823847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.831960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.833796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.834386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.834556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] PATH_INFO: `/v3/projects/3ef2567f3b4143bba7463a9e966150fe` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.835124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1ff8b-fa92-4c38-8487-162bb57281de admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.835879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef2567f3b4143bba7463a9e966150fe)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.837980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.844809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:14.847446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.847752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.848039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:14.849845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:14.852377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:14.871336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:14.884424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1078/2146] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:14.885687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:14.905225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97745a-0bfe-4448-b798-149e0517dde0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:14.945868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef2567f3b4143bba7463a9e966150fe', 'request_id': 'req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.958960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef2567f3b4143bba7463a9e966150fe', 'request_id': 'req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.968975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef2567f3b4143bba7463a9e966150fe', 'request_id': 'req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.988847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] The token cache is being invalidate because project 3ef2567f3b4143bba7463a9e966150fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:14.990475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.991408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bc3b518-2ee5-41b4-aa66-8d761ab6a859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:14.994038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1069/2147] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:14 2022] DELETE /identity/v3/projects/3ef2567f3b4143bba7463a9e966150fe => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:14.998565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba95be-5c7a-4352-8ed7-429c08864f48 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:14.999042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba95be-5c7a-4352-8ed7-429c08864f48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:14.999324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba95be-5c7a-4352-8ed7-429c08864f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.017809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c9bdc-0421-41e7-a142-aaf80cb30e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.018727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c9bdc-0421-41e7-a142-aaf80cb30e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.019135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c9bdc-0421-41e7-a142-aaf80cb30e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.019428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c9bdc-0421-41e7-a142-aaf80cb30e2b admin admin] PATH_INFO: `/v3/projects/c85800d659ab43ff9b16266f9f037fc2/users/bdb847efec494be8a5a89da456f54d3d/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.019864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6dba95be-5c7a-4352-8ed7-429c08864f48 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:15.021171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6dba95be-5c7a-4352-8ed7-429c08864f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.031836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1070/2148] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:15.040230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c9bdc-0421-41e7-a142-aaf80cb30e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85800d659ab43ff9b16266f9f037fc2, user_id=bdb847efec494be8a5a89da456f54d3d, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.041004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.042774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.042887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c9bdc-0421-41e7-a142-aaf80cb30e2b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.043625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.061862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1079/2149] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:14 2022] PUT /identity/v3/projects/c85800d659ab43ff9b16266f9f037fc2/users/bdb847efec494be8a5a89da456f54d3d/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:15.067091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.067446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.068226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.073171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:15.074413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.093087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:15.094290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.147073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1071/2150] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:15.151046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.160079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f9af-369b-4045-8d7a-21a03e055174 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.162499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1080/2151] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:15.171243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.171870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.172155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.172605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.173001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.173405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.174033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:15.174540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.176216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.189576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1072/2152] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:46:15.194432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.204797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53c85-42f5-446e-8ac5-bfc7d0001a04 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.210960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.211728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.212137 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.212552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.213321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.215352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.249200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:15.258252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1073/2153] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:15.258637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7321d-c914-4587-9551-fecc454e3b25 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.262315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.342878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.343455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.343740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.344022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.363178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589806e-5353-4442-9933-459f0060d0fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.365417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.367103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.382945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.383363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1081/2154] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:15.385639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.386079 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.386470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.387785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.391391 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.407068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:15.434675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1074/2155] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:15.438721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.453172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d290-e162-4d93-b226-53c29a439815 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.459829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.460854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.461231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.461587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.462587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.464886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.481163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1082/2156] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:15.485987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.498419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e91d88-960c-4f21-bf4e-425380e085a5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.506607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac6097-b904-4969-badd-3edaa2fd0ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.507306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac6097-b904-4969-badd-3edaa2fd0ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.507650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac6097-b904-4969-badd-3edaa2fd0ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.507937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac6097-b904-4969-badd-3edaa2fd0ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.508900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac6097-b904-4969-badd-3edaa2fd0ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.511288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac6097-b904-4969-badd-3edaa2fd0ab3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.532331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.538938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1075/2157] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:15.558899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ba69d-7e9e-4ffd-9cff-6be0ee4c2654 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.573020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.576203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.576553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.576840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] PATH_INFO: `/v3/projects/517d21fd933640ba9f1a2c4e27d8e70c/users/da22237bf7b34c19844e1c369559d75f/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.616592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=517d21fd933640ba9f1a2c4e27d8e70c, user_id=da22237bf7b34c19844e1c369559d75f, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.619830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:15.662704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.663170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.663268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.676408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1083/2158] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:15 2022] PUT /identity/v3/projects/517d21fd933640ba9f1a2c4e27d8e70c/users/da22237bf7b34c19844e1c369559d75f/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:15.720554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.722888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.728957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] MFA Rules not processed for user `da22237bf7b34c19844e1c369559d75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:15.730275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.874990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1076/2159] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:15.883776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6b5b6-8f91-45c0-b35f-efd02b4f44dc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:15.887851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:15.889602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:15.902049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:15.903254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:15.903655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:15.904019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] PATH_INFO: `/v3/users/ff970e1070c340d6b8d99cd59631cf50` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:15.908062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff970e1070c340d6b8d99cd59631cf50)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:15.914345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:16.012937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a076546-473b-4993-8965-0a2bec569df5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:16.030931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dee3d0-d6c1-4650-b6e2-382cd77caef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:16.031783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dee3d0-d6c1-4650-b6e2-382cd77caef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:16.032166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dee3d0-d6c1-4650-b6e2-382cd77caef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:16.032575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dee3d0-d6c1-4650-b6e2-382cd77caef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:16.039573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff970e1070c340d6b8d99cd59631cf50'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:16.042612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff970e1070c340d6b8d99cd59631cf50'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:16.049782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dee3d0-d6c1-4650-b6e2-382cd77caef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:16.058306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff970e1070c340d6b8d99cd59631cf50'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:16.058930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff970e1070c340d6b8d99cd59631cf50'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:16.060087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dee3d0-d6c1-4650-b6e2-382cd77caef8 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:16.079251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff970e1070c340d6b8d99cd59631cf50'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:16.080280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1077/2160] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:16.106557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:16.143909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1084/2161] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:15 2022] DELETE /identity/v3/users/ff970e1070c340d6b8d99cd59631cf50 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:16.147154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:16.147428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:16.147976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:16.196610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:16.199192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:16.322158 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1078/2162] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:16.330293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:16.331946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:16.420330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca5371-8450-4e90-b0a1-cc4749daac2f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:16.446296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:16.450147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:16.451157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:16.451618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:16.469740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:16.471896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:16.486282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1085/2163] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:16.900584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:16.902260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:16.936697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e19ac8-c379-47ca-98fa-90e2c977a910 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:16.942126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:16.943495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:16.944056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:16.944438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] PATH_INFO: `/v3/projects/43ce673eb46b486d9f8cc64a452e01a6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:16.945730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ce673eb46b486d9f8cc64a452e01a6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:16.948280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:16.960158 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:17.019375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ce673eb46b486d9f8cc64a452e01a6', 'request_id': 'req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.027312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ce673eb46b486d9f8cc64a452e01a6', 'request_id': 'req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.040779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ce673eb46b486d9f8cc64a452e01a6', 'request_id': 'req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.063705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] The token cache is being invalidate because project 43ce673eb46b486d9f8cc64a452e01a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:17.064526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.066717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.072600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1079/2164] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:16 2022] DELETE /identity/v3/projects/43ce673eb46b486d9f8cc64a452e01a6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:17.074061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:17.087428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d370a-1b2f-4c6b-a3ab-f975ff882aa6 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:17.177489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:17.179281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:17.179636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:17.179926 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] PATH_INFO: `/v3/users/8195de71cf65447cab14607aaff993ae` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:17.181943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8195de71cf65447cab14607aaff993ae)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:17.184037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:17.253126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8195de71cf65447cab14607aaff993ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.253843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8195de71cf65447cab14607aaff993ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.264645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8195de71cf65447cab14607aaff993ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.265429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8195de71cf65447cab14607aaff993ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.279038 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8195de71cf65447cab14607aaff993ae'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:17.323521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1086/2165] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:17 2022] DELETE /identity/v3/users/8195de71cf65447cab14607aaff993ae => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:17.897809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:17.899418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:17.925422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e66d3-23ff-47b1-b4fd-10227bb828d1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:18.068520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:18.070134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:18.071845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:18.072695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] PATH_INFO: `/v3/projects/426e1c49367541c9b3d1000e5b750e76` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:18.075026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426e1c49367541c9b3d1000e5b750e76)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:18.077156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:18.163398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426e1c49367541c9b3d1000e5b750e76', 'request_id': 'req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.171182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426e1c49367541c9b3d1000e5b750e76', 'request_id': 'req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.187745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426e1c49367541c9b3d1000e5b750e76', 'request_id': 'req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.203229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] The token cache is being invalidate because project 426e1c49367541c9b3d1000e5b750e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:18.203421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.204713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.206583 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1080/2166] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:17 2022] DELETE /identity/v3/projects/426e1c49367541c9b3d1000e5b750e76 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:18.211282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:18.214011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:18.231016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587aaafa-faf6-4287-8012-283ba34891a6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:18.240264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:18.328344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:18.329494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:18.329958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:18.330386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] PATH_INFO: `/v3/users/8c6bb70f2ac34ca69d0840889e974fef` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:18.333387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6bb70f2ac34ca69d0840889e974fef)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:18.335784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:18.391542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:18.393404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:18.413677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60b56f-d47a-4aa6-bbb2-d91a25d6fa06 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:18.421407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:18.422417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:18.422708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:18.422986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] PATH_INFO: `/v3/users/f43bfa3b0a2a4fa28ef2e80295fed484` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:18.424235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43bfa3b0a2a4fa28ef2e80295fed484)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:18.427192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6bb70f2ac34ca69d0840889e974fef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.427333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:18.428170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6bb70f2ac34ca69d0840889e974fef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.446331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6bb70f2ac34ca69d0840889e974fef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.448700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6bb70f2ac34ca69d0840889e974fef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.464237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6bb70f2ac34ca69d0840889e974fef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.475426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:18.512761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1087/2167] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:18 2022] DELETE /identity/v3/users/8c6bb70f2ac34ca69d0840889e974fef => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:18.522926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43bfa3b0a2a4fa28ef2e80295fed484'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.542096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43bfa3b0a2a4fa28ef2e80295fed484'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.542861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43bfa3b0a2a4fa28ef2e80295fed484'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.556140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43bfa3b0a2a4fa28ef2e80295fed484'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.557264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43bfa3b0a2a4fa28ef2e80295fed484'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:18.588358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1081/2168] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:18 2022] DELETE /identity/v3/users/f43bfa3b0a2a4fa28ef2e80295fed484 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:18.859944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:18.862734 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:18.952846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4355d8-5a37-40c5-8860-bae099ec39ca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.009549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:19.046131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.047494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.047938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.048971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] PATH_INFO: `/v3/projects/6dba70de72e24d1084bec5a8c2042a7f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.050458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dba70de72e24d1084bec5a8c2042a7f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.053637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.167396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dba70de72e24d1084bec5a8c2042a7f', 'request_id': 'req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.183990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dba70de72e24d1084bec5a8c2042a7f', 'request_id': 'req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.199333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dba70de72e24d1084bec5a8c2042a7f', 'request_id': 'req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.204770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.206389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:19.217889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] The token cache is being invalidate because project 6dba70de72e24d1084bec5a8c2042a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:19.218454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.219535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ebf97a7c-767d-4f7b-9166-b5fbc48237d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.221928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1088/2169] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:18 2022] DELETE /identity/v3/projects/6dba70de72e24d1084bec5a8c2042a7f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:19.227686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23760f27-c382-40ca-8cc1-25ecc1d02813 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.228072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23760f27-c382-40ca-8cc1-25ecc1d02813 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.228505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23760f27-c382-40ca-8cc1-25ecc1d02813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.241116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39c0af-16bd-492e-a7f0-c37358d3164e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.269566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-23760f27-c382-40ca-8cc1-25ecc1d02813 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:19.270877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-23760f27-c382-40ca-8cc1-25ecc1d02813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:19.355558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.359044 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.359917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.360170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] PATH_INFO: `/v3/projects/7a22ae4c671d4c828ea95651dd7ec9cd` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.385275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a22ae4c671d4c828ea95651dd7ec9cd)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.396671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.430938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1089/2170] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:19.438696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.439145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.439750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.480021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:19.481393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:19.486875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a22ae4c671d4c828ea95651dd7ec9cd', 'request_id': 'req-2f6189c0-03b4-47f1-b5c4-1aae88992759'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.503386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a22ae4c671d4c828ea95651dd7ec9cd', 'request_id': 'req-2f6189c0-03b4-47f1-b5c4-1aae88992759'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.516036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a22ae4c671d4c828ea95651dd7ec9cd', 'request_id': 'req-2f6189c0-03b4-47f1-b5c4-1aae88992759'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.533456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] The token cache is being invalidate because project 7a22ae4c671d4c828ea95651dd7ec9cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:19.533775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.534481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f6189c0-03b4-47f1-b5c4-1aae88992759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:19.539786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1082/2171] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:19 2022] DELETE /identity/v3/projects/7a22ae4c671d4c828ea95651dd7ec9cd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:19.541558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272b837-2049-4a2c-8291-b1aa5b326088 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.541855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272b837-2049-4a2c-8291-b1aa5b326088 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.542175 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272b837-2049-4a2c-8291-b1aa5b326088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.568613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e272b837-2049-4a2c-8291-b1aa5b326088 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:19.569860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e272b837-2049-4a2c-8291-b1aa5b326088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:19.623933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:19.646807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1090/2172] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:19.650869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.666123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59291836-882a-4a93-8818-d4dab8843bf3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.682070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.682674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.682960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.683238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.683978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:19.684368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.684494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1083/2173] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:19.685954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.693381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.693679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.693976 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.698478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:19.718973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1091/2174] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:19.723204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.734935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:19.736692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:19.745062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4d9c2-84be-41a9-950d-35a0e966f421 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.752390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.753073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.753359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.753642 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.754674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.756552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1084/2175] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:19.756688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.760796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.774253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29013597-592d-43d8-9093-fbf63f6f42b5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.782303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.783017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.783271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.783547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.784339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:19.784645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.787827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.804812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1092/2176] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:19.806003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.806117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1085/2177] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:19.806980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.819326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fcf4-c5ae-4f28-98ad-342c9f1a3efb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.909729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b239-80c2-46f3-a568-b881cc41f2bc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:19.923127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.923911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.924207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.924624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.925701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.928354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.928634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:19.929334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:19.929621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:19.929900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:19.930620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:19.933750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:19.962718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1093/2178] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:19.966938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.971481 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1086/2179] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:19.976813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:19.984726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011768f-5403-4a60-8cc0-6fdc48afb20b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.050189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ad294-49bc-4b7f-be24-af47b78d69e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.056449 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.057522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.057954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.058339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.059257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.062283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.072649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.073459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.073750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.074205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.074960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.076985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.088887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1087/2180] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:19 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:20.092211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1094/2181] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:20.093228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.094110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.109382 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f5888-32f3-48c8-a97c-18c5e943d807 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.111297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb92053-2961-4804-81b5-b81c329e0ed2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.117003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.117695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.117980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.118261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.118999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.120014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.120672 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.120923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.121322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.121675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.122696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.126880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.133280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1095/2182] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:20.137555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.153918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1088/2183] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:20.156872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45b0d-f66d-434d-9df1-86c20fd167d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.157818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.169062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747a6b-fd11-4750-aff9-56b727f2d8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.170610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747a6b-fd11-4750-aff9-56b727f2d8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.170799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023da405-3982-4fcb-bcc2-df746ef973ae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.171313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747a6b-fd11-4750-aff9-56b727f2d8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.171940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747a6b-fd11-4750-aff9-56b727f2d8d0 admin admin] PATH_INFO: `/v3/projects/c70c0ab9b5134d0781dea0e0abc02ac8/users/5b0a34631db94d1d9fd5e865de5b9156/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.176126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.177457 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.177745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.178022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.178751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.180395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.192799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1089/2184] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:20.196691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3747a6b-fd11-4750-aff9-56b727f2d8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70c0ab9b5134d0781dea0e0abc02ac8, user_id=5b0a34631db94d1d9fd5e865de5b9156, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.196854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.201022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3747a6b-fd11-4750-aff9-56b727f2d8d0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.203958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:20.218992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a61d7-7d09-4d0e-9cd6-faa8965fcb2d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.248144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1096/2185] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:20 2022] PUT /identity/v3/projects/c70c0ab9b5134d0781dea0e0abc02ac8/users/5b0a34631db94d1d9fd5e865de5b9156/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:20.252766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.253058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.253385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.302944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:20.304442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:20.336445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d905d-8a3e-4597-8df1-5196959b4cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.337857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d905d-8a3e-4597-8df1-5196959b4cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.338439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d905d-8a3e-4597-8df1-5196959b4cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.339005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d905d-8a3e-4597-8df1-5196959b4cd8 admin admin] PATH_INFO: `/v3/projects/be0426d85d7e4579ab010001c0efff74/users/4dae7ca96d8d42428b6dc3d783385764/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.347280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:20.369440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d905d-8a3e-4597-8df1-5196959b4cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0426d85d7e4579ab010001c0efff74, user_id=4dae7ca96d8d42428b6dc3d783385764, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.376115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d905d-8a3e-4597-8df1-5196959b4cd8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.399593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1097/2186] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:20.415433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1090/2187] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:20 2022] PUT /identity/v3/projects/be0426d85d7e4579ab010001c0efff74/users/4dae7ca96d8d42428b6dc3d783385764/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:20.415741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.415741 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:20.422302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.422690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.423068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.476820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbebf8-9374-4f0a-aac5-1de7ea58a18b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.497469 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] MFA Rules not processed for user `4dae7ca96d8d42428b6dc3d783385764`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:20.498910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:20.541324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8753dc-1f06-4e40-afe1-f1cf505caf77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.541954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8753dc-1f06-4e40-afe1-f1cf505caf77 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.542324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8753dc-1f06-4e40-afe1-f1cf505caf77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.542508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8753dc-1f06-4e40-afe1-f1cf505caf77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.602022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1091/2188] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:20.617752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:20.620024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:20.658268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8753dc-1f06-4e40-afe1-f1cf505caf77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.660053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8753dc-1f06-4e40-afe1-f1cf505caf77 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.674029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1098/2189] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:20.740420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3628d06-85fd-4cbc-830e-c441fbbab1d4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:20.751642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:20.753621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:20.755248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:20.755893 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:20.773222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:20.775813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:20.783990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:20.794645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1092/2190] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:22.273686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:22.274243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:22.274464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:22.375704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] MFA Rules not processed for user `bdb847efec494be8a5a89da456f54d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:22.378016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:22.403126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:22.576371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1099/2191] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:22.578778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:22.581894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:22.715545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c59a4d-4862-450c-a38a-37beecc76100 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:22.736737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:22.737599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:22.738601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:22.738601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:22.739209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:22.741323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:22.784026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1093/2192] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:22 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:22.789886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:22.814857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b553e-5e1d-445e-a7d9-f862c11fc234 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:22.943176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:22.943888 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:22.946128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:22.946544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:22.947682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:22.950487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:22.977528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1100/2193] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:22 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:22.982222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:23.012825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ad681-0c94-40e3-98c1-d5a24a3b303e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:23.028913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:23.030437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:23.030437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:23.030437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:23.034881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:23.039836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:23.073907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1094/2194] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:23.075367 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:23.115998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced22349-731d-4346-ab9f-250a7885a9b4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:23.123592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:23.124765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:23.125404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:23.125866 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] PATH_INFO: `/v3/projects/ca93026e5b7e450e851a230162a45408/users/d165790929f04ad79560baff5225704b/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:23.150353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca93026e5b7e450e851a230162a45408, user_id=d165790929f04ad79560baff5225704b, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:23.154260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:23.183625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1101/2195] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:23 2022] PUT /identity/v3/projects/ca93026e5b7e450e851a230162a45408/users/d165790929f04ad79560baff5225704b/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:23.188731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:23.210767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666e3f3-54f7-47ea-b2ec-63e5d4869698 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:23.333624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f428b11-1a0a-486b-8099-9e48cbcc46a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:23.334388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f428b11-1a0a-486b-8099-9e48cbcc46a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:23.334737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f428b11-1a0a-486b-8099-9e48cbcc46a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:23.335036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f428b11-1a0a-486b-8099-9e48cbcc46a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:23.335993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f428b11-1a0a-486b-8099-9e48cbcc46a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:23.338495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f428b11-1a0a-486b-8099-9e48cbcc46a3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:23.356754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1095/2196] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:23.364752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:23.394694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba0bad-be7f-4d30-b634-f7765a571f63 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:23.403772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:23.407487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:23.408046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:23.408424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] PATH_INFO: `/v3/projects/ca93026e5b7e450e851a230162a45408/users/d165790929f04ad79560baff5225704b/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:23.414707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca93026e5b7e450e851a230162a45408, user_id=d165790929f04ad79560baff5225704b, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:23.418582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:23.436128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1102/2197] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:23 2022] PUT /identity/v3/projects/ca93026e5b7e450e851a230162a45408/users/d165790929f04ad79560baff5225704b/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:28.356470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:28.357125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:28.357560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:28.443773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] MFA Rules not processed for user `5b0a34631db94d1d9fd5e865de5b9156`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:28.445267 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:28.530397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:28.535075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1096/2198] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:28.540453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:28.614083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:28.638285 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03803781-4cce-443a-8ffd-74723c0eec59 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:28.643448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:28.644376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:28.645072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:28.645072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:28.645736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:28.647823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:28.700863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:28.708454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1103/2199] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:28 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:28.714919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1957b8c-15f4-436d-b8f7-06d6ec7854be None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:28.819326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:28.819804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:28.820986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:28.821513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:28.823246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:28.826614 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:28.897008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1097/2200] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:28 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:28.903333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:28.922844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da5bad-eec4-4f1e-b1f5-359cb6e6a85d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:28.935909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:28.940480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:28.941210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:28.942228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:28.943358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:28.946478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:28.971388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1104/2201] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:28.975824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:28.987473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216abc5-625f-47ac-98c3-ea4d7eb30539 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:28.995551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:28.996708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:28.997805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:28.998228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] PATH_INFO: `/v3/projects/25d42e3d3a094377b459e21f70fd3455/users/f1eb82f3cf544e878ff395ebbd34a85a/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:29.024709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d42e3d3a094377b459e21f70fd3455, user_id=f1eb82f3cf544e878ff395ebbd34a85a, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:29.030856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:29.060180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1098/2202] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:28 2022] PUT /identity/v3/projects/25d42e3d3a094377b459e21f70fd3455/users/f1eb82f3cf544e878ff395ebbd34a85a/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:29.061526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:29.077556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb83d4-6d83-4112-8f36-6c63e1989b85 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:29.272129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:29.274528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:29.274959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:29.275474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:29.276362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:29.278998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:29.291271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1105/2203] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:29.297402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:29.310758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bfcbe-3a8c-4490-8036-5848f74920e0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:29.323686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776344f1-359d-4a6c-9020-effbf003f056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:29.324269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776344f1-359d-4a6c-9020-effbf003f056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:29.324666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776344f1-359d-4a6c-9020-effbf003f056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:29.325038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776344f1-359d-4a6c-9020-effbf003f056 admin admin] PATH_INFO: `/v3/projects/25d42e3d3a094377b459e21f70fd3455/users/f1eb82f3cf544e878ff395ebbd34a85a/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:29.327439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776344f1-359d-4a6c-9020-effbf003f056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d42e3d3a094377b459e21f70fd3455, user_id=f1eb82f3cf544e878ff395ebbd34a85a, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:29.330567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776344f1-359d-4a6c-9020-effbf003f056 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:29.351906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1099/2204] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:29 2022] PUT /identity/v3/projects/25d42e3d3a094377b459e21f70fd3455/users/f1eb82f3cf544e878ff395ebbd34a85a/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:29.855146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:29.883366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ab20-8fef-44f3-9a9b-a42eea316787 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:30.023062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:30.024048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:30.024272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:30.024600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] PATH_INFO: `/v3/users/4dae7ca96d8d42428b6dc3d783385764` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:30.029476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dae7ca96d8d42428b6dc3d783385764)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:30.031658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:30.126884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dae7ca96d8d42428b6dc3d783385764'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:30.136140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dae7ca96d8d42428b6dc3d783385764'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:30.151892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dae7ca96d8d42428b6dc3d783385764'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:30.152724 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dae7ca96d8d42428b6dc3d783385764'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:30.156673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:30.183550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dae7ca96d8d42428b6dc3d783385764'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:30.217096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1106/2205] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:29 2022] DELETE /identity/v3/users/4dae7ca96d8d42428b6dc3d783385764 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:30.221777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499b40f-e1c5-43aa-b0fa-70723abaab96 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:30.222292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499b40f-e1c5-43aa-b0fa-70723abaab96 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:30.222400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499b40f-e1c5-43aa-b0fa-70723abaab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:30.293202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f499b40f-e1c5-43aa-b0fa-70723abaab96 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:30.294464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f499b40f-e1c5-43aa-b0fa-70723abaab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:30.413127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1100/2206] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:30.420849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:30.422748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:30.654640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74943af-7d4f-4d8e-9ea8-9b11a13c2b2b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:30.661775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:30.662847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:30.663368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:30.664089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:30.679528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:30.682476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:30.706452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1107/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:30.823078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c568-b583-4adf-a196-4e6b2a42b5e8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:30.823531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c568-b583-4adf-a196-4e6b2a42b5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:30.824005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c568-b583-4adf-a196-4e6b2a42b5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:30.861774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1be9c568-b583-4adf-a196-4e6b2a42b5e8 None None] MFA Rules not processed for user `6f7edee75b2845e6bbfbc64574691fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:30.863397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1be9c568-b583-4adf-a196-4e6b2a42b5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:30.977789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:31.025999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1101/2208] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:32.170276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:32.171968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:32.202089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509bfa-5ebe-4383-a43a-586ea3eeca97 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:32.213827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:32.216781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:32.217180 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:32.217565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] PATH_INFO: `/v3/projects/be0426d85d7e4579ab010001c0efff74` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:32.222609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0426d85d7e4579ab010001c0efff74)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:32.229197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:32.344759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0426d85d7e4579ab010001c0efff74', 'request_id': 'req-98c915f9-b7f4-4dca-a22b-b17a6b4001db'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:32.355213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0426d85d7e4579ab010001c0efff74', 'request_id': 'req-98c915f9-b7f4-4dca-a22b-b17a6b4001db'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:32.368465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0426d85d7e4579ab010001c0efff74', 'request_id': 'req-98c915f9-b7f4-4dca-a22b-b17a6b4001db'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:32.396179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] The token cache is being invalidate because project be0426d85d7e4579ab010001c0efff74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:32.396602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:32.397606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-98c915f9-b7f4-4dca-a22b-b17a6b4001db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:32.401205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1108/2209] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:32 2022] DELETE /identity/v3/projects/be0426d85d7e4579ab010001c0efff74 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:32.406294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:32.407339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:32.408941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:32.536651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:32.544564 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:32.784164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1102/2210] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:32.793824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:32.794136 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:32.794516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:32.858050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:32.867943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:32.894373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1109/2211] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:32.899276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:32.912632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1600a68-ecec-446a-9c55-8d625786a21f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:32.931484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:32.932606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:32.933039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:32.933464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:32.934735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:32.935232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:32.939931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914f74f-d476-4f5d-9b7d-b36585797f9e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:32.971651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1103/2212] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:32.977144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:33.001830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64445005-4544-44a0-ad86-2549fb7e9c85 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:33.016761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.017046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.017337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.033847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:33.047141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.047396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.047634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.048427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:33.063180 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:33.094371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] MFA Rules not processed for user `d165790929f04ad79560baff5225704b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:33.096112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:33.132697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:33.155069 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1110/2213] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:32 2022] POST /identity/v3/projects => generated 403 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:33.168265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:33.286651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1104/2214] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:33.306823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:33.306823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:33.321908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a73114-24e7-4a4f-9eb4-89821dca41cf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:33.343340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:33.344098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.344456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.345380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.347735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:33.350818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:33.402878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1111/2215] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:33 2022] POST /identity/v3/users => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:33.407283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:33.431937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4cb5a-ee03-4622-93bc-fc710f05ea1a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:33.448185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:33.449124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.449398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.449680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.450300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:33.453918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:33.461370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:33.498390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1112/2216] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:33.503498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:33.514859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b1c6c-c9b5-4895-97e3-eea5d7f28830 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:33.531646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c5723-21eb-4baf-a2c4-6b1f00b3a705 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:33.552879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:33.553738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.554141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.554607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.555727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:33.562171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:33.580200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1113/2217] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:33.583538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:33.597732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f30475-b797-4e2e-b91d-05ac979ac46d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:33.605207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a24ad-c6ea-471d-b478-9df1c498c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:33.606573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a24ad-c6ea-471d-b478-9df1c498c5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.606961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a24ad-c6ea-471d-b478-9df1c498c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.620757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a24ad-c6ea-471d-b478-9df1c498c5ec admin admin] PATH_INFO: `/v3/projects/1af5d8c7a07743d6bdbf2dd77128a744/users/c49b6ef511124485bf7e674be477041a/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.674445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a24ad-c6ea-471d-b478-9df1c498c5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af5d8c7a07743d6bdbf2dd77128a744, user_id=c49b6ef511124485bf7e674be477041a, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:33.682083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a24ad-c6ea-471d-b478-9df1c498c5ec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:33.724389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1114/2218] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:33 2022] PUT /identity/v3/projects/1af5d8c7a07743d6bdbf2dd77128a744/users/c49b6ef511124485bf7e674be477041a/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:33.726717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.727180 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.727630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.765086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:33.765760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:33.766045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:33.766713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:33.769566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:33.853922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] MFA Rules not processed for user `c49b6ef511124485bf7e674be477041a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:33.855984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:33.960572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:33.961646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:33.998668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1105/2219] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:34.037915 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:34.040287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1115/2220] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:34.051856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3a17-5747-4772-8f2d-7891aa4fe5bf service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:34.141573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:34.142401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.142753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.143094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:34.146390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:34.155629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:34.240071 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1106/2221] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:34 2022] POST /identity/v3/projects => generated 405 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:34.241701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:34.258066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0a7d5-316d-4ab0-948c-f375123377a7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:34.341447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:34.399082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:34.399978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.400621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.400803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:34.401730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:34.404087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:34.439147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1116/2222] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:34 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:34.444129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:34.480394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f24f25-393b-46b2-b8d5-ff9665aee731 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:34.497764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:34.501938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.503715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.506509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:34.508780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:34.515894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:34.521714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:34.549716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1107/2223] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:34.554096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:34.570333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a8482-76d5-45e3-aadb-6c7597594214 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:34.585424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:34.586418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.586700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.586980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] PATH_INFO: `/v3/projects/8a9cf1170f91474d8eb5b0caeaa55ec4/users/d589d810a04b4680b36acbec3c4cb331/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:34.631579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9cf1170f91474d8eb5b0caeaa55ec4, user_id=d589d810a04b4680b36acbec3c4cb331, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:34.634372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:34.663262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1117/2224] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:34 2022] PUT /identity/v3/projects/8a9cf1170f91474d8eb5b0caeaa55ec4/users/d589d810a04b4680b36acbec3c4cb331/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:34.667492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:34.687644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a3fe9-00cc-4104-aba0-735a95bcbf97 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:34.856882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cf1ea-bc17-4575-8679-1acebba2caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:34.857590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf1ea-bc17-4575-8679-1acebba2caa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.857872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf1ea-bc17-4575-8679-1acebba2caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.858433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cf1ea-bc17-4575-8679-1acebba2caa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:34.859190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf1ea-bc17-4575-8679-1acebba2caa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:34.865315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cf1ea-bc17-4575-8679-1acebba2caa0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:34.885157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1108/2225] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:34.886540 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:34.926912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47977b08-9a4e-4e02-ba58-36cc564beb31 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:34.934081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:34.935315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.936714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.937187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] PATH_INFO: `/v3/projects/8a9cf1170f91474d8eb5b0caeaa55ec4/users/d589d810a04b4680b36acbec3c4cb331/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:34.940545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9cf1170f91474d8eb5b0caeaa55ec4, user_id=d589d810a04b4680b36acbec3c4cb331, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:34.945823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:34.970555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1118/2226] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:34 2022] PUT /identity/v3/projects/8a9cf1170f91474d8eb5b0caeaa55ec4/users/d589d810a04b4680b36acbec3c4cb331/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:34.989429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc989a5-0c9b-40e5-83bb-351b8f9606d3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:34.989769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc989a5-0c9b-40e5-83bb-351b8f9606d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:34.990628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc989a5-0c9b-40e5-83bb-351b8f9606d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:35.029021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5cc989a5-0c9b-40e5-83bb-351b8f9606d3 None None] MFA Rules not processed for user `d589d810a04b4680b36acbec3c4cb331`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:35.030270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5cc989a5-0c9b-40e5-83bb-351b8f9606d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:35.194871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1109/2227] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:35.233271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:35.235467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:35.376830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb93ef1-6b77-4182-8fa0-ef047161ef57 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:35.389723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:35.390296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:35.390638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:35.390927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:35.415265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:35.416808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:35.441097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1119/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:35 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:35.879431 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932d483-e733-47b5-ade6-de781de4c2e2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:35.879991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932d483-e733-47b5-ade6-de781de4c2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:35.880444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932d483-e733-47b5-ade6-de781de4c2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:35.923536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f932d483-e733-47b5-ade6-de781de4c2e2 None None] MFA Rules not processed for user `bdb847efec494be8a5a89da456f54d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:35.925393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f932d483-e733-47b5-ade6-de781de4c2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:36.063093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1110/2229] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:38.124961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:38.266336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0480-bc96-4167-b978-096158fae231 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:38.284654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:38.286244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:38.286767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:38.287128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] PATH_INFO: `/v3/users/c49b6ef511124485bf7e674be477041a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:38.288565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49b6ef511124485bf7e674be477041a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:38.290636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:38.363459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:38.397702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49b6ef511124485bf7e674be477041a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:38.398721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49b6ef511124485bf7e674be477041a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:38.420781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49b6ef511124485bf7e674be477041a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:38.433353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49b6ef511124485bf7e674be477041a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:38.454593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49b6ef511124485bf7e674be477041a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:38.523498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259666c-1cb0-4897-9e5c-6d132343d7e3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:38.525733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259666c-1cb0-4897-9e5c-6d132343d7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:38.526251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1120/2230] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:38 2022] DELETE /identity/v3/users/c49b6ef511124485bf7e674be477041a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:38.526463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259666c-1cb0-4897-9e5c-6d132343d7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:38.555089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c259666c-1cb0-4897-9e5c-6d132343d7e3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:38.556889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c259666c-1cb0-4897-9e5c-6d132343d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:38.609329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:38.739670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1111/2231] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:38.774936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:38.776925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:38.956547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f1e8a-feaf-4891-9b17-242a26c52c6d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:38.998795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:39.004282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:39.005642 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:39.006121 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:39.039693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:39.041230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:39.094165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1121/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:46:40.388621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:40.389317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:40.389317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:40.427722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] MFA Rules not processed for user `f1eb82f3cf544e878ff395ebbd34a85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:40.429676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:40.468492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:40.470517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:40.489811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e45e1-ae79-45a9-a75d-87752618333d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:40.495328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:40.496210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:40.496541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:40.496821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] PATH_INFO: `/v3/projects/1af5d8c7a07743d6bdbf2dd77128a744` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:40.504432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af5d8c7a07743d6bdbf2dd77128a744)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:40.507755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:40.654232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1112/2233] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:40.671485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:40.682217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:40.689700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af5d8c7a07743d6bdbf2dd77128a744', 'request_id': 'req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:40.713990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af5d8c7a07743d6bdbf2dd77128a744', 'request_id': 'req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:40.729568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af5d8c7a07743d6bdbf2dd77128a744', 'request_id': 'req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:40.758450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] The token cache is being invalidate because project 1af5d8c7a07743d6bdbf2dd77128a744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:40.758837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:40.759955 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:40.761822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1122/2234] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:40 2022] DELETE /identity/v3/projects/1af5d8c7a07743d6bdbf2dd77128a744 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:40.766757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:40.768482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:40.823085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673efb34-a67c-450d-9446-cfb0c3cfff8f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:40.851984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:40.902653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47189c92-2541-4488-8561-4783412efed3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:40.904030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189c92-2541-4488-8561-4783412efed3 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:40.904357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189c92-2541-4488-8561-4783412efed3 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:40.905418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189c92-2541-4488-8561-4783412efed3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:40.913527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ca69-ac3e-4d5e-a745-880087ccb5f2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:40.914109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47189c92-2541-4488-8561-4783412efed3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:40.930373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:40.933662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:40.934157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:40.934653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] PATH_INFO: `/v3/users/d589d810a04b4680b36acbec3c4cb331` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:40.938210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d589d810a04b4680b36acbec3c4cb331)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:40.945654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:40.948029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:41.047827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd589d810a04b4680b36acbec3c4cb331'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:41.048744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd589d810a04b4680b36acbec3c4cb331'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:41.054133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189c92-2541-4488-8561-4783412efed3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:41.056279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189c92-2541-4488-8561-4783412efed3 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:41.061829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd589d810a04b4680b36acbec3c4cb331'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:41.062855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd589d810a04b4680b36acbec3c4cb331'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:41.075605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd589d810a04b4680b36acbec3c4cb331'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:41.088554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1113/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:41.106041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1123/2236] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:40 2022] DELETE /identity/v3/users/d589d810a04b4680b36acbec3c4cb331 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:41.267628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:41.269584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:41.271776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:41.335636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:41.337533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:41.481710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1114/2237] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:41.571553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:41.574079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:41.749017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bdbfb-ead6-4d52-b699-06bf707e2959 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:41.755017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:41.756628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:41.758593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fb72-372d-4be4-92f6-c4085cc75822 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:41.759421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fb72-372d-4be4-92f6-c4085cc75822 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:41.759895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fb72-372d-4be4-92f6-c4085cc75822 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:41.760187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fb72-372d-4be4-92f6-c4085cc75822 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:41.775720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fb72-372d-4be4-92f6-c4085cc75822 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:41.776964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fb72-372d-4be4-92f6-c4085cc75822 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:41.828107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1124/2238] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:46:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:46:41.977589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35e822-c505-42ba-bd0d-8b9a83a596ee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:41.983328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:41.984717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:41.985265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:41.985753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] PATH_INFO: `/v3/projects/8a9cf1170f91474d8eb5b0caeaa55ec4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:41.987446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9cf1170f91474d8eb5b0caeaa55ec4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:41.997991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:42.141905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9cf1170f91474d8eb5b0caeaa55ec4', 'request_id': 'req-bf884aa6-12f0-4bb8-8222-b259950af343'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:42.161350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9cf1170f91474d8eb5b0caeaa55ec4', 'request_id': 'req-bf884aa6-12f0-4bb8-8222-b259950af343'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:42.164785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:42.175479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9cf1170f91474d8eb5b0caeaa55ec4', 'request_id': 'req-bf884aa6-12f0-4bb8-8222-b259950af343'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:42.192602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] The token cache is being invalidate because project 8a9cf1170f91474d8eb5b0caeaa55ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:42.192905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:42.193761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf884aa6-12f0-4bb8-8222-b259950af343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:42.195753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1115/2239] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:41 2022] DELETE /identity/v3/projects/8a9cf1170f91474d8eb5b0caeaa55ec4 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:42.203247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:42.203572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:42.204388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:42.268693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:42.271961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:42.340488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:42.441950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1125/2240] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:42.455997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:42.456290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:42.460106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:42.486571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:42.488461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:42.498311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1116/2241] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:42.502542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:42.522733 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23136fb3-1f21-41d0-a320-e399e2ccc76b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:42.529126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:42.529825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:42.530113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:42.530397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:42.531146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:42.531510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:42.533781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:42.555862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1126/2242] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:42.560424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:42.575097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77362aa4-2a58-4abb-b777-49420aede06d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:42.586359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:42.587091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:42.587402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:42.587687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:42.588492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:42.590548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:42.658209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1117/2243] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:42 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:42.663715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:42.692978 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40800b90-76b1-4675-a1b4-a5d6f3bd461b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:42.928655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:42.929918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:42.930213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:42.930494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:42.931223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:42.933727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:42.989333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1127/2244] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:42 2022] POST /identity/v3/users => generated 287 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:42.993628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:43.006375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1552d8e-a954-418e-9a1d-b89b8f9b67f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:43.034803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:43.036390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:43.036697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:43.037009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:43.037863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:43.043835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:43.092337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:43.093941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1118/2245] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:43.107406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516cd69-8c87-4a37-99a5-7e293d2864fa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:43.133967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0903-49ad-409a-a50d-d476852149a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:43.135023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0903-49ad-409a-a50d-d476852149a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:43.135321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0903-49ad-409a-a50d-d476852149a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:43.135621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0903-49ad-409a-a50d-d476852149a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:43.136426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff0903-49ad-409a-a50d-d476852149a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:43.138158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff0903-49ad-409a-a50d-d476852149a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:43.171982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1128/2246] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:43.176083 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:43.190888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed69ed4-d996-4ee2-a29a-e7238f2a3037 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:43.211095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:43.212058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:43.212418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:43.212655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] PATH_INFO: `/v3/projects/435b7cd9b71d41a9b84f12519f530452/users/05ccc86db5904d259a1746de0530e777/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:43.264553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=435b7cd9b71d41a9b84f12519f530452, user_id=05ccc86db5904d259a1746de0530e777, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:43.267601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:43.293031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1119/2247] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:43 2022] PUT /identity/v3/projects/435b7cd9b71d41a9b84f12519f530452/users/05ccc86db5904d259a1746de0530e777/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:43.307569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:43.307854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:43.308146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:43.402333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] MFA Rules not processed for user `05ccc86db5904d259a1746de0530e777`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:43.404585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:43.580552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:43.581132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1129/2248] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:43.716455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e275acf-af1c-4957-8474-299bc0284d6e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:43.724604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:43.725399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:43.725782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:43.726165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:43.727153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:43.729403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:43.792623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:43.802879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1120/2249] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:43 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:43.811943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badd8fc-5ff3-4b10-a40e-3b9aac92f5d1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:43.882839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:43.948129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:43.949211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:43.949581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:43.949956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:43.951026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:43.953128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:43.985645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1130/2250] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:43 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:46:43.992196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:44.012677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81495342-dc38-4c69-9fa2-2b886c4f21a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:44.018266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:44.020021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.020775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.021160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:44.022236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:44.033777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:44.099098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1121/2251] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:44.103212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:44.144730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6edd0e-b57a-4aec-9c9d-4c8a0c34b420 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:44.182172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:44.185955 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.186135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.186427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] PATH_INFO: `/v3/projects/1a574ff5e2b447afb9264ef740b37d16/users/b5e97f66e4f24f779b0a9198c9d2ac87/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:44.254571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a574ff5e2b447afb9264ef740b37d16, user_id=b5e97f66e4f24f779b0a9198c9d2ac87, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:44.266712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:44.336505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:44.338752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:44.340620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1131/2252] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:44 2022] PUT /identity/v3/projects/1a574ff5e2b447afb9264ef740b37d16/users/b5e97f66e4f24f779b0a9198c9d2ac87/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:44.353747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:44.402836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacac6b1-e1d4-4087-9856-0f2956fcc558 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:44.492818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49bf4b-01be-4dfa-85c2-e55ff79f8f33 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:44.500224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:44.501007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.501298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.501577 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:44.502366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:44.505032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:44.547706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1132/2253] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:44.554547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:44.576541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7822ba-5edb-4d4a-80e3-a045d9bac33e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:44.583284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708add76-b597-42e6-b82a-57288e3027fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:44.585848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708add76-b597-42e6-b82a-57288e3027fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.586296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708add76-b597-42e6-b82a-57288e3027fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.587175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708add76-b597-42e6-b82a-57288e3027fe admin admin] PATH_INFO: `/v3/projects/1a574ff5e2b447afb9264ef740b37d16/users/b5e97f66e4f24f779b0a9198c9d2ac87/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:44.593955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:44.599608 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708add76-b597-42e6-b82a-57288e3027fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a574ff5e2b447afb9264ef740b37d16, user_id=b5e97f66e4f24f779b0a9198c9d2ac87, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:44.608944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708add76-b597-42e6-b82a-57288e3027fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:44.618204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:44.619119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.619406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.619692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] PATH_INFO: `/v3/users/da22237bf7b34c19844e1c369559d75f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:44.624796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=da22237bf7b34c19844e1c369559d75f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:44.629673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1133/2254] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:44 2022] PUT /identity/v3/projects/1a574ff5e2b447afb9264ef740b37d16/users/b5e97f66e4f24f779b0a9198c9d2ac87/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:44.633870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:44.647221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.647587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.648062 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:44.729695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] MFA Rules not processed for user `b5e97f66e4f24f779b0a9198c9d2ac87`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:44.734120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:44.783740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da22237bf7b34c19844e1c369559d75f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:44.836949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da22237bf7b34c19844e1c369559d75f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:44.839212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da22237bf7b34c19844e1c369559d75f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:44.885966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da22237bf7b34c19844e1c369559d75f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:44.892961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c011def-24ca-4f54-89dd-975af502a1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da22237bf7b34c19844e1c369559d75f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:44.923513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:44.933186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1134/2255] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:44.951333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:44.955096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:44.983291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1122/2256] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:46:44 2022] DELETE /identity/v3/users/da22237bf7b34c19844e1c369559d75f => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:46:44.988737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:44.989320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:44.991899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:45.045701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:45.047211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:45.188114 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1123/2257] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:45.254917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996979b-a979-4723-b2cc-e6e541311dae None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:45.260439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:45.262017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:45.405135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:45.406176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:45.406522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:45.406919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:45.456223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe62ce4-9280-4be9-bb92-921042737fd2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:45.468729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:45.471409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:45.472986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:45.476220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:45.517016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:45.518722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:45.558576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1124/2258] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:45.591945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:45.596377 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:45.640793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1135/2259] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:45.803479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:45.825693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6f27-21f9-499f-83dd-67f0d0319f0a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:45.831008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:45.833066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:45.833655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:45.834166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:45.844351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:45.882355 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:46.061239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:46.074424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d351ac7-a82a-459d-8855-150c7786b724 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:46.101548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1125/2260] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:47.042981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:47.045999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:47.088683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b746654-210a-402a-9d4c-cc2d85b21774 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:47.124013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:47.124963 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:47.127072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:47.127342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] PATH_INFO: `/v3/projects/517d21fd933640ba9f1a2c4e27d8e70c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:47.131359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517d21fd933640ba9f1a2c4e27d8e70c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:47.133245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:47.296982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517d21fd933640ba9f1a2c4e27d8e70c', 'request_id': 'req-0b032731-18b3-487d-84c0-4bfc74d09039'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:47.349230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517d21fd933640ba9f1a2c4e27d8e70c', 'request_id': 'req-0b032731-18b3-487d-84c0-4bfc74d09039'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:47.375436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517d21fd933640ba9f1a2c4e27d8e70c', 'request_id': 'req-0b032731-18b3-487d-84c0-4bfc74d09039'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:47.377291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:47.401264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] The token cache is being invalidate because project 517d21fd933640ba9f1a2c4e27d8e70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:46:47.401566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:47.405275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0b032731-18b3-487d-84c0-4bfc74d09039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:46:47.410984 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1136/2261] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:46:47 2022] DELETE /identity/v3/projects/517d21fd933640ba9f1a2c4e27d8e70c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:47.412908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:47.413300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:47.413403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:47.441952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:47.445334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:47.596797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1126/2262] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:47.607376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:47.608066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:47.608610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:47.671147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:47.671904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:47.704401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1137/2263] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:47.707168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:47.728416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800227be-770a-4535-ae50-1bdcd5fb7245 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:47.746422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:47.747159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:47.747450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:47.747748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:47.749626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:46:47.749962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:47.756163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:47.798980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1127/2264] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:46:47.804982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:47.841211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44badaa-f75d-4e51-a3d8-b0bd064ef807 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:47.862476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:47.868230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:47.868541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:47.870131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:47.873181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:47.887380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:47.969937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1138/2265] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:47 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:47.975786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:47.991801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b433e7-d89f-477e-b67a-fbf25dfe034e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:48.184226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:48.185146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:48.185574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:48.185998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:48.186957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:48.189237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:48.255503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1128/2266] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:47 2022] POST /identity/v3/users => generated 292 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:48.257307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:48.275014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3e8b7-8625-41a7-867d-41180bfdba34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:48.293194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:48.294542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:48.295250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:48.295784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:48.296950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:48.305272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:48.342480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1139/2267] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:46:48.343233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:48.361569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612571d-8583-4462-aa5d-126709b88f8d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:48.370310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167f80d-4dda-4214-8ffa-6def358f57c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:48.372801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167f80d-4dda-4214-8ffa-6def358f57c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:48.373094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167f80d-4dda-4214-8ffa-6def358f57c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:48.373377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167f80d-4dda-4214-8ffa-6def358f57c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:48.376166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167f80d-4dda-4214-8ffa-6def358f57c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:48.376166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167f80d-4dda-4214-8ffa-6def358f57c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:48.406122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1129/2268] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:48.409143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:48.421149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2274c6f-4bfd-4a77-9110-032125a3cc49 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:48.426848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:48.428081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:48.428403 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:48.428686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] PATH_INFO: `/v3/projects/6a07a074bd38429e97a8f4a7ff2bbcf8/users/c60fbe05793f47e5afa0b3a43ea4983b/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:48.452012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a07a074bd38429e97a8f4a7ff2bbcf8, user_id=c60fbe05793f47e5afa0b3a43ea4983b, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:48.456010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:48.477806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1140/2269] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:48 2022] PUT /identity/v3/projects/6a07a074bd38429e97a8f4a7ff2bbcf8/users/c60fbe05793f47e5afa0b3a43ea4983b/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:48.490374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:48.490975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:48.491182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:48.527089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] MFA Rules not processed for user `c60fbe05793f47e5afa0b3a43ea4983b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:48.528770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:48.548789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:48.692358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1130/2270] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:48.695908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:48.822682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1070b-5f98-47bc-b97a-5d87c30be6e0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:48.827960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:48.830042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:48.830835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:48.830835 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:48.831750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:48.834251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:48.896347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1141/2271] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:48 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:48.900373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:48.919433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f4611-d5f2-4c14-bc40-9c6f179e8204 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:49.152487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:49.153358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:49.153749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:49.154130 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:49.154948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:49.157368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:49.198702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1131/2272] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:48 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:49.202480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:49.231769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3c8d-c681-428d-b3ce-c03f68178cfb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:49.266831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:49.269912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:49.270812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:49.271307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:49.273109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:49.276076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:49.294331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1142/2273] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:49.298292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:49.327658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba990fc-37c2-4c7e-b4b4-0498cee8ba43 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:49.337527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6fdc9-d41b-45a0-8f58-f5cb7daf6783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:49.338239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6fdc9-d41b-45a0-8f58-f5cb7daf6783 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:49.338524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6fdc9-d41b-45a0-8f58-f5cb7daf6783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:49.338804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6fdc9-d41b-45a0-8f58-f5cb7daf6783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:49.339604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6fdc9-d41b-45a0-8f58-f5cb7daf6783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:49.341300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6fdc9-d41b-45a0-8f58-f5cb7daf6783 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:49.394795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1132/2274] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:49.407912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:49.463671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74d7e5-a417-4e98-8a76-3ccaf861e848 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:49.483858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:49.484873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:49.485158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:49.485437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] PATH_INFO: `/v3/projects/d9bd33dcf0e5423583a8ea858edd291b/users/b948e166eaf7487c89e443ecff8e6a96/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:49.540091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bd33dcf0e5423583a8ea858edd291b, user_id=b948e166eaf7487c89e443ecff8e6a96, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:49.542824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:49.564373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1143/2275] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:49 2022] PUT /identity/v3/projects/d9bd33dcf0e5423583a8ea858edd291b/users/b948e166eaf7487c89e443ecff8e6a96/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:49.577361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:49.577361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:49.577361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:49.611905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] MFA Rules not processed for user `b948e166eaf7487c89e443ecff8e6a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:49.613425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:49.800274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1133/2276] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:46:49.805089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:49.904546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:49.959196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c655182-8c67-4450-b774-c45748f75129 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:49.969604 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:49.974487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:49.976695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:49.978767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:49.979804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:49.985086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:50.074854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1144/2277] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:46:49 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:50.081713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:50.132494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cb987-d846-48f3-ae3e-230e147710a9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:50.204093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:50.350293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:50.352575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:50.353126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:50.353749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:50.354595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:50.361637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:50.426323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1134/2278] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:46:50 2022] POST /identity/v3/users => generated 299 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:46:50.428023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:50.455536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244464-74f4-4e69-a6da-e7a11fb81bd2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:50.486509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:50.487215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:50.487576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:50.488384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:50.489220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:50.490778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:50.552902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1145/2279] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:50.554784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:50.572424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0233e-d5e4-4d7a-83cb-8c6e085a3f7e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:50.590972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:50.597027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:50.597027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:50.597027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] PATH_INFO: `/v3/projects/eeb4907256c04adb91c2d5abae2b1f25/users/72bbebe431fa420fb362baff3c717f4b/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:50.627143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb4907256c04adb91c2d5abae2b1f25, user_id=72bbebe431fa420fb362baff3c717f4b, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:50.633094 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:50.658024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1135/2280] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:50 2022] PUT /identity/v3/projects/eeb4907256c04adb91c2d5abae2b1f25/users/72bbebe431fa420fb362baff3c717f4b/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:50.659674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:50.675781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18afdd6-afe4-4304-ad9c-e7b3cb385287 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:50.840932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69784-d673-41c3-84bc-b8b93c00a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:50.841989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69784-d673-41c3-84bc-b8b93c00a64e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:50.842910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69784-d673-41c3-84bc-b8b93c00a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:50.843348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69784-d673-41c3-84bc-b8b93c00a64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:50.844685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69784-d673-41c3-84bc-b8b93c00a64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:50.846786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69784-d673-41c3-84bc-b8b93c00a64e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:50.867446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1146/2281] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:46:50.871553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:50.887526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2741f-31e0-42bf-9a44-d4bfea670a2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:50.898596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:50.899634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:50.899999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:50.900381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] PATH_INFO: `/v3/projects/eeb4907256c04adb91c2d5abae2b1f25/users/72bbebe431fa420fb362baff3c717f4b/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:50.905129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb4907256c04adb91c2d5abae2b1f25, user_id=72bbebe431fa420fb362baff3c717f4b, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:50.908062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:50.937528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1136/2282] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:46:50 2022] PUT /identity/v3/projects/eeb4907256c04adb91c2d5abae2b1f25/users/72bbebe431fa420fb362baff3c717f4b/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:46:50.951858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5070-7936-4782-8c88-e19700038c5d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:50.952284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5070-7936-4782-8c88-e19700038c5d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:50.952635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5070-7936-4782-8c88-e19700038c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:50.986197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-407d5070-7936-4782-8c88-e19700038c5d None None] MFA Rules not processed for user `72bbebe431fa420fb362baff3c717f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:50.987788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-407d5070-7936-4782-8c88-e19700038c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:51.099602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1147/2283] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:51.111982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:51.116109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:51.228625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdc4c-ae7b-4880-956c-7dc6d4af9943 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:51.236110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:51.236110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:51.236110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:51.236563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:51.325006 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:51.326522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:51.340033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1137/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:57.045560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:57.046111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:57.046220 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:57.074301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] MFA Rules not processed for user `5b0a34631db94d1d9fd5e865de5b9156`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:46:57.075547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:57.114231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:57.150616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1148/2285] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:46:57.486807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:57.489195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:46:57.533461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96e8fc-900c-4187-bb91-27f1e4737503 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:57.537960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565ec52-67a2-4996-8e90-b3a90b194a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:57.538643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565ec52-67a2-4996-8e90-b3a90b194a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:57.538930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565ec52-67a2-4996-8e90-b3a90b194a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:57.539210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565ec52-67a2-4996-8e90-b3a90b194a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:57.545295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:46:57.614887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565ec52-67a2-4996-8e90-b3a90b194a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:57.616447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565ec52-67a2-4996-8e90-b3a90b194a00 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:57.626468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1138/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:46:58.385212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:46:58.396673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399d06-88b0-4fbb-a695-00d0a33a8a8a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:46:58.400485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:46:58.401071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:46:58.401357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:46:58.401635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:46:58.412451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:46:58.413915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:46:58.423915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1149/2287] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:46:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:02.355281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:02.356669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:02.357393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:02.464848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:02.466902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:02.700773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1139/2288] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:02.722418 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:02.726207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:02.847397 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1bfbb-c98d-4c5e-87d0-33cd29c0b21b service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:02.853954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b2242-be5f-4f78-80ba-d895e9c39663 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:02.855387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b2242-be5f-4f78-80ba-d895e9c39663 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:02.855387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b2242-be5f-4f78-80ba-d895e9c39663 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:02.855387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b2242-be5f-4f78-80ba-d895e9c39663 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:02.866759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b2242-be5f-4f78-80ba-d895e9c39663 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:02.868719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b2242-be5f-4f78-80ba-d895e9c39663 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:02.879200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1150/2289] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:47:03.722219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:03.901990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffac90-d8b5-4924-a969-a30a9ad0310d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:03.920545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:03.925534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:03.926208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:03.926704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] PATH_INFO: `/v3/users/c60fbe05793f47e5afa0b3a43ea4983b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:03.928378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60fbe05793f47e5afa0b3a43ea4983b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:03.935776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:04.093300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60fbe05793f47e5afa0b3a43ea4983b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:04.094896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:04.127970 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60fbe05793f47e5afa0b3a43ea4983b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:04.128991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60fbe05793f47e5afa0b3a43ea4983b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:04.140517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60fbe05793f47e5afa0b3a43ea4983b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:04.142733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60fbe05793f47e5afa0b3a43ea4983b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:04.182001 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1140/2290] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:03 2022] DELETE /identity/v3/users/c60fbe05793f47e5afa0b3a43ea4983b => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:04.184901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:04.185444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:04.185927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:04.273143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:04.277803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:04.418628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1151/2291] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:04.430213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:04.432555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:04.592029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1dbfd-f563-4ea0-b487-bfcd35a78d38 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:04.603724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:04.605425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:04.606310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:04.607076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:04.637823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:04.643206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:04.660139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1141/2292] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:05.085306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:05.087176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:05.108809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27746466-2a21-4981-8b90-fd2ea402c6c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:05.115024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:05.115951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:05.116290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:05.116760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] PATH_INFO: `/v3/projects/6a07a074bd38429e97a8f4a7ff2bbcf8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:05.117869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a07a074bd38429e97a8f4a7ff2bbcf8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:05.119841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:05.202806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:05.205067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a07a074bd38429e97a8f4a7ff2bbcf8', 'request_id': 'req-6beaa466-15c4-4f99-9978-f000b19ea374'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.219941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a07a074bd38429e97a8f4a7ff2bbcf8', 'request_id': 'req-6beaa466-15c4-4f99-9978-f000b19ea374'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.237387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a07a074bd38429e97a8f4a7ff2bbcf8', 'request_id': 'req-6beaa466-15c4-4f99-9978-f000b19ea374'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.254814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] The token cache is being invalidate because project 6a07a074bd38429e97a8f4a7ff2bbcf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:05.255301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.256414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.259074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1152/2293] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:05 2022] DELETE /identity/v3/projects/6a07a074bd38429e97a8f4a7ff2bbcf8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:47:05.264947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:05.282556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32d003-093d-47d2-a139-1d090d9d0df1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:05.395961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:05.399316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:05.400493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:05.403049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] PATH_INFO: `/v3/users/b948e166eaf7487c89e443ecff8e6a96` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:05.405817 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b948e166eaf7487c89e443ecff8e6a96)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:05.410070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:05.531913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b948e166eaf7487c89e443ecff8e6a96'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.558110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b948e166eaf7487c89e443ecff8e6a96'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.559357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b948e166eaf7487c89e443ecff8e6a96'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.568602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:05.580050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b948e166eaf7487c89e443ecff8e6a96'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.582368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b948e166eaf7487c89e443ecff8e6a96'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:05.630879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1142/2294] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:05 2022] DELETE /identity/v3/users/b948e166eaf7487c89e443ecff8e6a96 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:47:06.355487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:06.358150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:06.410237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa466-15c4-4f99-9978-f000b19ea374 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:06.532207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:06.533801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:06.535033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:06.536092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] PATH_INFO: `/v3/projects/d9bd33dcf0e5423583a8ea858edd291b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:06.538093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9bd33dcf0e5423583a8ea858edd291b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:06.541016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:06.619717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9bd33dcf0e5423583a8ea858edd291b', 'request_id': 'req-9940062c-ad39-469f-b8d1-777b3e4941c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.634081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9bd33dcf0e5423583a8ea858edd291b', 'request_id': 'req-9940062c-ad39-469f-b8d1-777b3e4941c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.640631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9bd33dcf0e5423583a8ea858edd291b', 'request_id': 'req-9940062c-ad39-469f-b8d1-777b3e4941c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.675602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] The token cache is being invalidate because project d9bd33dcf0e5423583a8ea858edd291b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:06.675871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.676541 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.678181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1153/2295] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:06 2022] DELETE /identity/v3/projects/d9bd33dcf0e5423583a8ea858edd291b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:06.683911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:06.685522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:06.703328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3040-7f97-4c4a-a858-dfdc3ac216cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:06.784294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:06.784942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:06.785237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:06.785525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] PATH_INFO: `/v3/users/72bbebe431fa420fb362baff3c717f4b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:06.786838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=72bbebe431fa420fb362baff3c717f4b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:06.788923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:06.907466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72bbebe431fa420fb362baff3c717f4b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.937029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72bbebe431fa420fb362baff3c717f4b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.938598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72bbebe431fa420fb362baff3c717f4b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.950187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72bbebe431fa420fb362baff3c717f4b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.951195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c62a4c1a-6cbb-442e-9b0d-0b20a54a3eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72bbebe431fa420fb362baff3c717f4b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:06.979190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1143/2296] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:06 2022] DELETE /identity/v3/users/72bbebe431fa420fb362baff3c717f4b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:07.401867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:07.403561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:07.452596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940062c-ad39-469f-b8d1-777b3e4941c9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:07.583861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:07.585016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:07.585340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:07.585799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] PATH_INFO: `/v3/projects/eeb4907256c04adb91c2d5abae2b1f25` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:07.587553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb4907256c04adb91c2d5abae2b1f25)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:07.590709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:07.603765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:07.705648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb4907256c04adb91c2d5abae2b1f25', 'request_id': 'req-386b5e5d-6e12-42c1-8085-f7e701c05427'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:07.727027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb4907256c04adb91c2d5abae2b1f25', 'request_id': 'req-386b5e5d-6e12-42c1-8085-f7e701c05427'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:07.749120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb4907256c04adb91c2d5abae2b1f25', 'request_id': 'req-386b5e5d-6e12-42c1-8085-f7e701c05427'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:07.808521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] The token cache is being invalidate because project eeb4907256c04adb91c2d5abae2b1f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:07.809067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:07.810014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-386b5e5d-6e12-42c1-8085-f7e701c05427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:07.812452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1154/2297] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:07 2022] DELETE /identity/v3/projects/eeb4907256c04adb91c2d5abae2b1f25 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:07.823010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:07.823747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:07.823747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:07.891331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:07.892865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:08.024734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1144/2298] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:08.035226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.035525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.035806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.097160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:08.101169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:08.143133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1155/2299] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:08.144677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:08.176866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ded360-ebb3-4cd4-a5f7-dbef98776278 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:08.218303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:08.218303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.218627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.218724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.219504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:47:08.220076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:08.222877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:08.224669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:08.255707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1145/2300] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:47:08.260110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:08.272326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4321b-a4f6-4ff7-8ff3-3e5dd6982da9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:08.287521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:08.288638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.296715 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.297359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.298327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:08.301369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:08.367858 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1156/2301] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:47:08 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:08.368900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:08.391717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992b749-b62e-49c7-9501-29fba3d66767 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:08.521928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b428-2541-4130-926a-f4743669758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:08.522548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b428-2541-4130-926a-f4743669758c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.522841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b428-2541-4130-926a-f4743669758c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.523125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b428-2541-4130-926a-f4743669758c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.523842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b428-2541-4130-926a-f4743669758c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:08.529014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b428-2541-4130-926a-f4743669758c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:08.580993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1146/2302] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:47:08 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:08.585100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:08.600812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529b022-e23d-415e-b1f2-67da00b2c5d9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:08.616657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:08.617805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.618295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.618774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.628411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:08.633646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:08.650921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1157/2303] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:08.655311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b428-2541-4130-926a-f4743669758c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:08.672751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b428-2541-4130-926a-f4743669758c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:08.678841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14454a51-e885-4553-92ce-3dccf487c30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:08.679540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14454a51-e885-4553-92ce-3dccf487c30d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.679827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14454a51-e885-4553-92ce-3dccf487c30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.680112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14454a51-e885-4553-92ce-3dccf487c30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.681098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14454a51-e885-4553-92ce-3dccf487c30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:08.682746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14454a51-e885-4553-92ce-3dccf487c30d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:08.694826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1147/2304] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:08.700254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:08.730359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef51cf5-64fa-48e2-84b7-d02a9759411b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:08.736195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:08.738995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.739573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.740071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] PATH_INFO: `/v3/projects/7b4298aa065642c0bcfaed146905ce85/users/2cd8ed5659b74354bd590f8fa2d5e19a/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.765250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4298aa065642c0bcfaed146905ce85, user_id=2cd8ed5659b74354bd590f8fa2d5e19a, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:08.769280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:08.786728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1158/2305] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:47:08 2022] PUT /identity/v3/projects/7b4298aa065642c0bcfaed146905ce85/users/2cd8ed5659b74354bd590f8fa2d5e19a/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:08.798209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1bf7d-e315-4de9-8345-7b68b8f72cea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:08.798567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1bf7d-e315-4de9-8345-7b68b8f72cea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:08.799516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1bf7d-e315-4de9-8345-7b68b8f72cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:08.836194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5af1bf7d-e315-4de9-8345-7b68b8f72cea None None] MFA Rules not processed for user `2cd8ed5659b74354bd590f8fa2d5e19a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:08.841214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5af1bf7d-e315-4de9-8345-7b68b8f72cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:08.999448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1148/2306] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:47:09.015067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:09.017408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:09.137817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7335ab-7337-4999-b6cb-a469aed51eea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:09.149511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1aa81-4dc7-416a-852d-b2e3eda1fc44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:09.150875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1aa81-4dc7-416a-852d-b2e3eda1fc44 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:09.151163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1aa81-4dc7-416a-852d-b2e3eda1fc44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:09.151456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1aa81-4dc7-416a-852d-b2e3eda1fc44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:09.165497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1aa81-4dc7-416a-852d-b2e3eda1fc44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:09.167059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1aa81-4dc7-416a-852d-b2e3eda1fc44 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:09.182630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1159/2307] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:47:10.930372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:10.931193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:10.931193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:10.950900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] MFA Rules not processed for user `2cd8ed5659b74354bd590f8fa2d5e19a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:10.952159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:10.960424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1149/2308] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:47:19.173326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:20.944726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:31.777169 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:31.777693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:31.777788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:31.833130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:31.834363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:31.905795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1160/2309] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:47:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:31.922395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:31.924152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:32.042490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2220a8-1615-4bea-b780-abaef8e0ade1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:32.050753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208348ec-d127-42f7-8f64-37623f9f86ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:32.052983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208348ec-d127-42f7-8f64-37623f9f86ee service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:32.053281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208348ec-d127-42f7-8f64-37623f9f86ee service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:32.053645 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208348ec-d127-42f7-8f64-37623f9f86ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:32.065371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208348ec-d127-42f7-8f64-37623f9f86ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:32.067024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208348ec-d127-42f7-8f64-37623f9f86ee service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:32.076374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1150/2310] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:47:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:47:41.902649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=40 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:42.072635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=41 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:44.470603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:44.472415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:44.555726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfae202-e10c-4df2-8e2d-b191ee243156 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:44.560691 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:44.561667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:44.562045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:44.562244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] PATH_INFO: `/v3/users/05ccc86db5904d259a1746de0530e777` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:44.563625 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ccc86db5904d259a1746de0530e777)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:44.565842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:44.640147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ccc86db5904d259a1746de0530e777'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:44.640968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ccc86db5904d259a1746de0530e777'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:44.649168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ccc86db5904d259a1746de0530e777'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:44.650305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ccc86db5904d259a1746de0530e777'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:44.661504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ccc86db5904d259a1746de0530e777'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:44.693394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1161/2311] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:44 2022] DELETE /identity/v3/users/05ccc86db5904d259a1746de0530e777 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:44.698937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:44.699409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:44.699673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:44.723493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:44.724901 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:44.810183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1151/2312] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:44.820015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:44.822976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:44.993499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c7df0-43f7-4a82-b062-8227c0838979 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:45.001851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:45.002529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:45.002815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:45.003094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:45.013583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:45.015217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:45.089660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1162/2313] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:45.839778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:45.841707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:45.854303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1be45-6675-4daa-bfa3-59e63f139909 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:45.859371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:45.860407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:45.860688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:45.861537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] PATH_INFO: `/v3/projects/435b7cd9b71d41a9b84f12519f530452` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:45.864128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435b7cd9b71d41a9b84f12519f530452)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:45.867014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:45.929226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435b7cd9b71d41a9b84f12519f530452', 'request_id': 'req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:45.941748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435b7cd9b71d41a9b84f12519f530452', 'request_id': 'req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:45.959955 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435b7cd9b71d41a9b84f12519f530452', 'request_id': 'req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:45.979938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] The token cache is being invalidate because project 435b7cd9b71d41a9b84f12519f530452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:45.980226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:45.980920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:45.984147 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1152/2314] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:45 2022] DELETE /identity/v3/projects/435b7cd9b71d41a9b84f12519f530452 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:47:45.988014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:45.999122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229df6dd-1fc3-4d18-a992-2ec3cfac5c8a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:46.024557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:46.073340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:46.074346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:46.074607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:46.074957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] PATH_INFO: `/v3/users/b5e97f66e4f24f779b0a9198c9d2ac87` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:46.076148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e97f66e4f24f779b0a9198c9d2ac87)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:46.078273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:46.147941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e97f66e4f24f779b0a9198c9d2ac87'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:46.148810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e97f66e4f24f779b0a9198c9d2ac87'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:46.158270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e97f66e4f24f779b0a9198c9d2ac87'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:46.158981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e97f66e4f24f779b0a9198c9d2ac87'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:46.168429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e97f66e4f24f779b0a9198c9d2ac87'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:46.221479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1163/2315] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:45 2022] DELETE /identity/v3/users/b5e97f66e4f24f779b0a9198c9d2ac87 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:46.270308 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:46.272227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:46.469828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e6ae5-5cd7-4e1a-8f2c-e712a82f0f23 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:46.475949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:46.666321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:46.667501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:46.668665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:46.669188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:46.676833 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:46.712567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:46.714999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91527705-3eb3-4d45-b3c3-af61942a7549 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:46.755688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1153/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:47.009103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.010868 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:47.100163 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2ee24-e0e6-4d5e-ab0e-1cd3abce4e37 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.106713 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.107648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.107964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.108290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] PATH_INFO: `/v3/projects/1a574ff5e2b447afb9264ef740b37d16` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.119423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a574ff5e2b447afb9264ef740b37d16)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.121536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:47.195299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:47.212143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a574ff5e2b447afb9264ef740b37d16', 'request_id': 'req-3446b487-b4ec-4a0e-bbae-2b0b1d081724'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:47.231673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a574ff5e2b447afb9264ef740b37d16', 'request_id': 'req-3446b487-b4ec-4a0e-bbae-2b0b1d081724'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:47.241637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a574ff5e2b447afb9264ef740b37d16', 'request_id': 'req-3446b487-b4ec-4a0e-bbae-2b0b1d081724'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:47.256923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] The token cache is being invalidate because project 1a574ff5e2b447afb9264ef740b37d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:47.257619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:47.258478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3446b487-b4ec-4a0e-bbae-2b0b1d081724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:47.260578 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1164/2317] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:47 2022] DELETE /identity/v3/projects/1a574ff5e2b447afb9264ef740b37d16 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:47:47.265629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.266341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.266341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.306659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:47.311444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:47.445596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1154/2318] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:47:47.454335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.455142 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.455627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.483816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:47.486019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:47.499937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1165/2319] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:47.501215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.518029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83077d00-9527-43cc-9d0c-fd28f27db129 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.524975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.526133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.526596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.527095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.528187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:47:47.528667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.531030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:47.549703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1155/2320] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:47:47.551174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.562070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e2541-0f52-42b4-ac15-6f15fe02ad15 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.568242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.569118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.569511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.570050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.570846 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.573473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:47.618310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1166/2321] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:47:47 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:47.622652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.645465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9fc3a-d9b7-4563-9664-8e469bae0f2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.778730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.779570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.779966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.780424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.782129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.784706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:47.823486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1156/2322] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:47:47 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:47.825042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.838439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f35fa8-d147-4748-9d1f-c2d2ee301a73 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.843461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.844774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.845250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.845742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.846878 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.849465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:47.865446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1167/2323] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:47.869326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.883963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ba086-88ed-4608-bc91-e3ba33d28c42 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.894254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c30bc-cbd0-4e89-9cdd-6963079e65e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.895259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c30bc-cbd0-4e89-9cdd-6963079e65e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.895764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c30bc-cbd0-4e89-9cdd-6963079e65e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.896230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c30bc-cbd0-4e89-9cdd-6963079e65e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.897193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c30bc-cbd0-4e89-9cdd-6963079e65e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.900838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c30bc-cbd0-4e89-9cdd-6963079e65e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:47.916763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1157/2324] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:47.924433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:47.935152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bb79-b58b-4c1d-8b85-d84720dd2a02 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:47.947345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:47.952267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:47.953735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:47.954031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] PATH_INFO: `/v3/projects/98b6ce08db6744b9b9d85efde207c5fc/users/ebd7c24e926f43258258f6be12187b2b/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:47.972425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b6ce08db6744b9b9d85efde207c5fc, user_id=ebd7c24e926f43258258f6be12187b2b, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:47.975453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:48.010026 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1168/2325] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:47:47 2022] PUT /identity/v3/projects/98b6ce08db6744b9b9d85efde207c5fc/users/ebd7c24e926f43258258f6be12187b2b/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:48.024841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8848b-254f-4940-9665-1bd84172314c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:48.025372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8848b-254f-4940-9665-1bd84172314c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:48.026814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8848b-254f-4940-9665-1bd84172314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:48.041110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:48.083279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-afe8848b-254f-4940-9665-1bd84172314c None None] MFA Rules not processed for user `ebd7c24e926f43258258f6be12187b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:48.085003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afe8848b-254f-4940-9665-1bd84172314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:48.215673 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1158/2326] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:48.228567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:48.230539 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:48.302135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d3a42-d07e-441e-9a98-894b80e440d3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:48.305987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f014e-7959-472c-8bc6-9e5412b15308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:48.306651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f014e-7959-472c-8bc6-9e5412b15308 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:48.306934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f014e-7959-472c-8bc6-9e5412b15308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:48.307211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f014e-7959-472c-8bc6-9e5412b15308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:48.319134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f014e-7959-472c-8bc6-9e5412b15308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:48.320844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f014e-7959-472c-8bc6-9e5412b15308 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:48.354781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1169/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:48.362088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8848b-254f-4940-9665-1bd84172314c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:48.451232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8848b-254f-4940-9665-1bd84172314c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:48.457138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:48.458535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:48.459025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:48.459515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] PATH_INFO: `/v3/users/ebd7c24e926f43258258f6be12187b2b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:48.461761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd7c24e926f43258258f6be12187b2b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:48.464023 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:48.543166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd7c24e926f43258258f6be12187b2b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:48.565666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd7c24e926f43258258f6be12187b2b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:48.566534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd7c24e926f43258258f6be12187b2b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:48.579324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd7c24e926f43258258f6be12187b2b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:48.580088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd7c24e926f43258258f6be12187b2b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:48.595339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1159/2328] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:48 2022] DELETE /identity/v3/users/ebd7c24e926f43258258f6be12187b2b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:47:48.599814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:48.600022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:48.600338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:48.627170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:48.628443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:48.720341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1170/2329] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:48.729424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:48.731128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:48.747156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:48.846139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b1865-1603-4fdb-8e76-af329f4d4e27 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:48.851529 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf2061-0160-434a-8082-41fd0e21ce6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:48.852369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf2061-0160-434a-8082-41fd0e21ce6d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:48.852663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf2061-0160-434a-8082-41fd0e21ce6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:48.853193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf2061-0160-434a-8082-41fd0e21ce6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:48.864801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf2061-0160-434a-8082-41fd0e21ce6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:48.866532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf2061-0160-434a-8082-41fd0e21ce6d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:48.883873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1160/2330] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:47:49.707411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:49.708947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:49.745823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493e9ca-20de-4b2d-bd30-4dcc8a49a80f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:49.772236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:49.773433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:49.773726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:49.774007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] PATH_INFO: `/v3/projects/98b6ce08db6744b9b9d85efde207c5fc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:49.776943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b6ce08db6744b9b9d85efde207c5fc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:49.784021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:49.799651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:49.868478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b6ce08db6744b9b9d85efde207c5fc', 'request_id': 'req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:49.883546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b6ce08db6744b9b9d85efde207c5fc', 'request_id': 'req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:49.891812 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b6ce08db6744b9b9d85efde207c5fc', 'request_id': 'req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:49.906158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] The token cache is being invalidate because project 98b6ce08db6744b9b9d85efde207c5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:49.906510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:49.907950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1fe3712-b19a-4bd0-872a-b78f8a26b66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:49.910193 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1171/2331] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:49 2022] DELETE /identity/v3/projects/98b6ce08db6744b9b9d85efde207c5fc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:49.914894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:49.915675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:49.915675 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:49.936815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:49.939025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:50.019507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1161/2332] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:50.027161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.027462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.027753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.047987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:50.049598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:50.058823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1172/2333] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:50.062844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.073117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a871d-0aa6-4923-9d68-82bdfe6f96b5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.078075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.079767 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.080129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.080521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.081347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:47:50.081718 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.083412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.097784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1162/2334] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:47:50.102152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.112576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b778-7822-4e65-aef2-c6929cb4fae5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.118109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.118444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.118535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.118798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.119473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.121600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.163507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1173/2335] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:47:50 2022] POST /identity/v3/projects => generated 359 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:50.167994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.180365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b45183-ca11-4c77-a26b-8bce3fc5873c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.324902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.326999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.327933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.328522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.329474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.332029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.366480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1163/2336] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:47:50 2022] POST /identity/v3/users => generated 280 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:50.370466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.384824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b99a-b297-4c1d-ac28-e4bb8d9ac4f6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.390966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.392192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.392839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.393126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.393921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.395510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.417796 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1174/2337] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:50.421829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.434806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11df1be-1388-4e4e-9dff-64d666f93e03 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.443463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff7eb4-a980-474e-9fee-b5b51f4cce97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.444151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff7eb4-a980-474e-9fee-b5b51f4cce97 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.444524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff7eb4-a980-474e-9fee-b5b51f4cce97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.444841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff7eb4-a980-474e-9fee-b5b51f4cce97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.445619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff7eb4-a980-474e-9fee-b5b51f4cce97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.447430 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff7eb4-a980-474e-9fee-b5b51f4cce97 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.466786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1164/2338] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:50.468281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.479518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa0b9-0ba8-452d-869c-84fcd83dee61 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.484001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.485674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.485935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.486214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] PATH_INFO: `/v3/projects/bd9faec66dc4425999df313f155bda87/users/9a8c78d482ab440296bda2ccfca35f31/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.501759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9faec66dc4425999df313f155bda87, user_id=9a8c78d482ab440296bda2ccfca35f31, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.504674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.519781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1175/2339] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:47:50 2022] PUT /identity/v3/projects/bd9faec66dc4425999df313f155bda87/users/9a8c78d482ab440296bda2ccfca35f31/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:50.530340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.530479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.530739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.559383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] MFA Rules not processed for user `9a8c78d482ab440296bda2ccfca35f31`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:50.560835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:50.569612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:50.639909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1165/2340] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:50.677442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.679487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:50.747745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f54c3-ba78-48f4-84d7-fd138c99327f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.753075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd2175-c6a5-4cbd-8992-075b28ff2197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.756906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd2175-c6a5-4cbd-8992-075b28ff2197 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.756906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd2175-c6a5-4cbd-8992-075b28ff2197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.756906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd2175-c6a5-4cbd-8992-075b28ff2197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.766388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd2175-c6a5-4cbd-8992-075b28ff2197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.767918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd2175-c6a5-4cbd-8992-075b28ff2197 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.779035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1176/2341] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:50 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:50.793643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:50.858750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0fa3e-f6bd-4a92-ba7e-35aca7387f76 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:50.865628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:50.866531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.867007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.867523 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] PATH_INFO: `/v3/users/9a8c78d482ab440296bda2ccfca35f31` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.869193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8c78d482ab440296bda2ccfca35f31)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:50.871200 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:50.925609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8c78d482ab440296bda2ccfca35f31'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:50.938296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8c78d482ab440296bda2ccfca35f31'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:50.939179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8c78d482ab440296bda2ccfca35f31'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:50.947320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8c78d482ab440296bda2ccfca35f31'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:50.948258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8c78d482ab440296bda2ccfca35f31'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:50.960844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1166/2342] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:50 2022] DELETE /identity/v3/users/9a8c78d482ab440296bda2ccfca35f31 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:50.965411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573443d-8708-4d6e-be7d-b11e2dda460d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:50.965707 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573443d-8708-4d6e-be7d-b11e2dda460d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:50.966010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573443d-8708-4d6e-be7d-b11e2dda460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:50.983471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1573443d-8708-4d6e-be7d-b11e2dda460d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:50.985443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1573443d-8708-4d6e-be7d-b11e2dda460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.014659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:51.049072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1177/2343] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:51.059138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:51.060649 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.097236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:51.130115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03462-874e-4f14-9955-6499566eb672 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:51.133775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:51.134423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.134702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.134977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.144837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:51.146277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:51.156237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1167/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:51.297358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.298219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.298786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.326003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:51.327273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.407018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1178/2345] 10.222.0.16 () {58 vars in 967 bytes} [Thu May 26 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:51.422171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:51.423521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.500910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aafc29-8c30-4a26-9ddd-4947697608e0 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:51.506049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601a18-afd1-4c50-b0bb-6d5c9e999a9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:51.506712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601a18-afd1-4c50-b0bb-6d5c9e999a9c service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.507002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601a18-afd1-4c50-b0bb-6d5c9e999a9c service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.507284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601a18-afd1-4c50-b0bb-6d5c9e999a9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.516998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601a18-afd1-4c50-b0bb-6d5c9e999a9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:51.518632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601a18-afd1-4c50-b0bb-6d5c9e999a9c service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:51.527956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1168/2346] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:47:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:47:51.560673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:51.562841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.584254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebaf14-1e02-4122-88b0-75b8ccca6a6c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:51.588840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:51.589754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.590045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.590328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] PATH_INFO: `/v3/projects/bd9faec66dc4425999df313f155bda87` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.591530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9faec66dc4425999df313f155bda87)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:51.593523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:51.654584 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9faec66dc4425999df313f155bda87', 'request_id': 'req-f8db71b2-390c-42b7-ab83-985e89044905'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:51.663024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9faec66dc4425999df313f155bda87', 'request_id': 'req-f8db71b2-390c-42b7-ab83-985e89044905'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:51.669806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9faec66dc4425999df313f155bda87', 'request_id': 'req-f8db71b2-390c-42b7-ab83-985e89044905'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:51.682525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] The token cache is being invalidate because project bd9faec66dc4425999df313f155bda87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:51.683059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:51.683485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f8db71b2-390c-42b7-ab83-985e89044905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:51.685104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1179/2347] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:51 2022] DELETE /identity/v3/projects/bd9faec66dc4425999df313f155bda87 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:51.691494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.691789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.692141 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.711838 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:51.713163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.800195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1169/2348] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:51.809040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.809500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.809688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.839004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:51.840802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:51.874335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1180/2349] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:51.878578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:51.891283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6941d-500f-405d-8f13-861648cf9b60 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:51.897013 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:51.898359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.898656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.899014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.899816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:47:51.900131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:51.904486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:51.940041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1170/2350] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:47:51.940847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:51.954693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fe65b-8971-47dc-9621-acccebd69ed9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:51.961644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:51.962571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:51.963071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:51.963428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:51.964340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:51.967005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:52.010134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1181/2351] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:47:51 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:52.012074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:52.025168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6249df-3808-486e-975d-214e6566f52f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:52.092260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:52.110154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:52.110769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:52.110922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:52.111209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:52.111912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:52.114508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:52.142868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1171/2352] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:47:52 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:52.147013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:52.158234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41437d-2e23-43e2-aaa3-68796df5c151 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:52.163469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:52.164166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:52.164597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:52.164893 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:52.165631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:52.167253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:52.180389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1182/2353] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:52.184719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:52.207035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea25f5-b6b9-4f3a-a5a1-6fcd9218357e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:52.213830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdffae9-77ba-4da3-a0bd-33cc353e78a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:52.214632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdffae9-77ba-4da3-a0bd-33cc353e78a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:52.214865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdffae9-77ba-4da3-a0bd-33cc353e78a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:52.215154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdffae9-77ba-4da3-a0bd-33cc353e78a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:52.215895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdffae9-77ba-4da3-a0bd-33cc353e78a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:52.217555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdffae9-77ba-4da3-a0bd-33cc353e78a2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:52.229854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1172/2354] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:52.234213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:52.244935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd185575-b403-4642-b507-e90b15a86bfb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:52.249829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:52.251282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:52.251564 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:52.251848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] PATH_INFO: `/v3/projects/990a5207faa8471dba30671255d1ebfa/users/afe536ef1c804099bfec9a075bd42392/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:52.266917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990a5207faa8471dba30671255d1ebfa, user_id=afe536ef1c804099bfec9a075bd42392, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:52.269676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:52.278134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:52.286361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1183/2355] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:47:52 2022] PUT /identity/v3/projects/990a5207faa8471dba30671255d1ebfa/users/afe536ef1c804099bfec9a075bd42392/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:47:52.297368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:52.297682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:52.297963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:52.330182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] MFA Rules not processed for user `afe536ef1c804099bfec9a075bd42392`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:52.331545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:52.400823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1173/2356] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:52.413072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:52.415178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:52.494704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c60b0-d374-486d-b3bf-117369ed2e88 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:52.498688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:52.499342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:52.499632 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:52.500077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:52.508275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:52.509735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:52.518912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1184/2357] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:47:53.773070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:53.774631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:53.849846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d17807-2767-4fa0-8767-f3e485af3764 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:53.854480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:53.855364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:53.855651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:53.855928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] PATH_INFO: `/v3/users/6f7edee75b2845e6bbfbc64574691fb0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:53.857219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7edee75b2845e6bbfbc64574691fb0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:53.859207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:53.914971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7edee75b2845e6bbfbc64574691fb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:53.925571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:53.931795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7edee75b2845e6bbfbc64574691fb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:53.932634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7edee75b2845e6bbfbc64574691fb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:53.941159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7edee75b2845e6bbfbc64574691fb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:53.942268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-67b9caaa-1ae9-4750-8dd4-19a29ad3fe27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7edee75b2845e6bbfbc64574691fb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:53.964233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1174/2358] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:47:53 2022] DELETE /identity/v3/users/6f7edee75b2845e6bbfbc64574691fb0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:54.231958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.253825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeede76f-17ce-4885-97e3-ca8b7e4633ca service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:54.352431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:54.353308 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.353596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.353876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] PATH_INFO: `/v3/projects/c879e95c508f401fbc77e63b1e64dd16` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.355261 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c879e95c508f401fbc77e63b1e64dd16)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:54.357256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:54.412009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c879e95c508f401fbc77e63b1e64dd16', 'request_id': 'req-bc99fdc2-721a-4108-a6bd-576679dd73a6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:54.419894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c879e95c508f401fbc77e63b1e64dd16', 'request_id': 'req-bc99fdc2-721a-4108-a6bd-576679dd73a6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:54.427389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c879e95c508f401fbc77e63b1e64dd16', 'request_id': 'req-bc99fdc2-721a-4108-a6bd-576679dd73a6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:54.440494 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] The token cache is being invalidate because project c879e95c508f401fbc77e63b1e64dd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:47:54.441076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:54.442089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bc99fdc2-721a-4108-a6bd-576679dd73a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:47:54.443988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1185/2359] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:47:54 2022] DELETE /identity/v3/projects/c879e95c508f401fbc77e63b1e64dd16 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:54.450908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.451207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.451496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.470177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:54.471465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:54.541542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1175/2360] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:54.549994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.550290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.550722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.579370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:54.580465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:54.588886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1186/2361] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:54.593413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.604043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a26806-644f-4767-bf11-e3b3df3a47cf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:54.608907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:54.610121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.610451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.610839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.611587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:47:54.611906 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:54.613710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:54.632800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1176/2362] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:47:54.638267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.655196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:54.667493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d8e9d-4e6e-462e-aef1-6d684a9c8498 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:54.709394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:54.712797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.713311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.714044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.716532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:54.719356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:54.806206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1187/2363] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:47:54 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:54.810237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.821961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29238fa1-07ba-4cac-a67b-874265c8e492 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:54.894968 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:54.895671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.895957 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.896236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.897037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:54.899155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:54.929787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1177/2364] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:47:54 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:47:54.934189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.954757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a5143-b699-4bc3-88ab-e82e565cb54e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:54.962399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:54.963390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:54.964445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:54.964445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:54.965335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:54.967046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:54.983301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1188/2365] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:54.984837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:54.998812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f495f-9320-492f-bb93-8ca55e878708 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:55.005113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91564527-beeb-42a3-9972-7bf0813c8765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:55.006159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564527-beeb-42a3-9972-7bf0813c8765 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:55.006466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564527-beeb-42a3-9972-7bf0813c8765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:55.006827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564527-beeb-42a3-9972-7bf0813c8765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:55.007591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91564527-beeb-42a3-9972-7bf0813c8765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:55.010237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91564527-beeb-42a3-9972-7bf0813c8765 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:55.028339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1178/2366] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:47:55.029918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:55.041135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075f396-c452-4bfa-bcf8-4307b9057bf3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:55.048530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:55.049464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:55.049753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:55.050055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] PATH_INFO: `/v3/projects/1b57613abfd645d687bfe07afd8b3d32/users/a88a4c921cd74128afcd6c23342d8d15/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:55.070656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b57613abfd645d687bfe07afd8b3d32, user_id=a88a4c921cd74128afcd6c23342d8d15, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:55.073780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:55.091708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1189/2367] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:47:55 2022] PUT /identity/v3/projects/1b57613abfd645d687bfe07afd8b3d32/users/a88a4c921cd74128afcd6c23342d8d15/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:47:55.108697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:55.110107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:55.110687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:55.142282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] MFA Rules not processed for user `a88a4c921cd74128afcd6c23342d8d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:47:55.143986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:55.178316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:47:55.221034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1179/2368] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:47:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:47:55.232596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:55.235170 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:55.298442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec3bfb-0c9c-48bc-a881-17016d984748 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:55.303320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295297c-b45d-41de-8319-49e6bcae6629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:55.304126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295297c-b45d-41de-8319-49e6bcae6629 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:55.304505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295297c-b45d-41de-8319-49e6bcae6629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:55.304866 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295297c-b45d-41de-8319-49e6bcae6629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:55.315165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295297c-b45d-41de-8319-49e6bcae6629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:55.317009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295297c-b45d-41de-8319-49e6bcae6629 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:55.327293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1190/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:47:58.338174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:47:58.340394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:58.426144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62387ec3-12a9-4018-8cb5-52c039e8c9f5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:47:58.430217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:47:58.430979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:47:58.432055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:47:58.432055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:47:58.433827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:47:58.509145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:47:58.511801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:47:58.548447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1180/2370] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:47:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:03.489936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909960d4-0e49-4f55-9757-16f4c837aa2a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:03.490502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909960d4-0e49-4f55-9757-16f4c837aa2a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:03.490605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909960d4-0e49-4f55-9757-16f4c837aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:03.525073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-909960d4-0e49-4f55-9757-16f4c837aa2a None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:03.527582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-909960d4-0e49-4f55-9757-16f4c837aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:03.600483 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1191/2371] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:48:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:03.612910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:03.614788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:03.703693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16472a57-2976-4396-8cb5-9d8a8df858d7 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:03.708690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:03.709361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:03.709626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:03.710033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:03.719372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:03.721157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:03.731362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1181/2372] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:48:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:48:04.303818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf8a4d-a77f-4fce-9d40-06fa910db6aa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:04.304113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf8a4d-a77f-4fce-9d40-06fa910db6aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:04.304459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf8a4d-a77f-4fce-9d40-06fa910db6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:04.327052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c2cf8a4d-a77f-4fce-9d40-06fa910db6aa None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:04.328693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c2cf8a4d-a77f-4fce-9d40-06fa910db6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:04.336610 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1192/2373] 10.222.0.16 () {58 vars in 967 bytes} [Thu May 26 13:48:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:04.350652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:04.364461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f7d91-f0b2-422a-93ba-ee97a11ec91b service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:04.371803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b9a94-0eb6-47c3-9946-9a9f8f789fa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:04.372579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b9a94-0eb6-47c3-9946-9a9f8f789fa8 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:04.372964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b9a94-0eb6-47c3-9946-9a9f8f789fa8 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:04.373314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b9a94-0eb6-47c3-9946-9a9f8f789fa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:04.383509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b9a94-0eb6-47c3-9946-9a9f8f789fa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:04.385238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b9a94-0eb6-47c3-9946-9a9f8f789fa8 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:04.389625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:04.396342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1182/2374] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 13:48:04.604777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb0144-cfaa-4758-a2e2-ad8a250263a4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:04.605460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb0144-cfaa-4758-a2e2-ad8a250263a4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:04.605460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb0144-cfaa-4758-a2e2-ad8a250263a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:04.629885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fedb0144-cfaa-4758-a2e2-ad8a250263a4 None None] MFA Rules not processed for user `dd8055e46e6149d3897fb88af0b19d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:04.631503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fedb0144-cfaa-4758-a2e2-ad8a250263a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:04.661222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:04.661784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:04.661784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:04.682360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:04.683591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:04.684599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:04.692252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1183/2375] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:48:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:04.698588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1193/2376] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:48:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:04.707479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:04.722985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205ff82-b683-424f-97eb-17edf5909ad6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:04.728220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:04.728898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:04.729186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:04.729473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:04.739160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:04.740821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:04.748905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1184/2377] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:48:05.111244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:05.111531 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:05.111813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:05.130937 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:05.132165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:05.139265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1194/2378] 10.222.0.16 () {58 vars in 967 bytes} [Thu May 26 13:48:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:05.151800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:05.163076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8992c-44a6-4298-803a-582723dd4cdc service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:05.168401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:05.169270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:05.169559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:05.169843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:05.178724 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:05.180388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:05.189693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1185/2379] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:48:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:48:09.512551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:09.514044 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:09.526866 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c312a0c-6203-4232-b79c-5f74e738e41a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:09.531212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:09.531864 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:09.532156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:09.532472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:09.534237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:09.599740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:09.601213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af85f4-e2bf-4677-96a2-b8a6225987c9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:09.610678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1195/2380] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:48:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:10.477747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:10.479265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:10.548661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6db85-9a72-4fe9-9425-a34741e8cded service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:10.553475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:10.554348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:10.554630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:10.554909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] PATH_INFO: `/v3/users/2cd8ed5659b74354bd590f8fa2d5e19a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:10.556109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd8ed5659b74354bd590f8fa2d5e19a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:10.558100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:10.620692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd8ed5659b74354bd590f8fa2d5e19a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:10.634629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd8ed5659b74354bd590f8fa2d5e19a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:10.636031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd8ed5659b74354bd590f8fa2d5e19a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:10.646142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd8ed5659b74354bd590f8fa2d5e19a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:10.646940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd8ed5659b74354bd590f8fa2d5e19a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:10.659850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1186/2381] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:48:10 2022] DELETE /identity/v3/users/2cd8ed5659b74354bd590f8fa2d5e19a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:48:10.665223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:10.665767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:10.666100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:10.684053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:10.685428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:10.753005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1196/2382] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:10.762484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:10.764010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:10.821534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:10.837004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11935a-8358-47d9-b879-d56d4806fcde admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:10.842030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:10.842775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:10.843154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:10.843500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:10.854563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:10.856136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:10.867510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1187/2383] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:48:11.095580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:11.102276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:11.127804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bdb52-9cfa-40db-9710-7b887a099344 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:11.133437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:11.135212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:11.135668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:11.135827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] PATH_INFO: `/v3/projects/7b4298aa065642c0bcfaed146905ce85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:11.137700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4298aa065642c0bcfaed146905ce85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:11.139588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:11.206717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4298aa065642c0bcfaed146905ce85', 'request_id': 'req-aa284908-4cb9-4465-a913-8e2117a68947'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:11.222973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4298aa065642c0bcfaed146905ce85', 'request_id': 'req-aa284908-4cb9-4465-a913-8e2117a68947'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:11.230841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4298aa065642c0bcfaed146905ce85', 'request_id': 'req-aa284908-4cb9-4465-a913-8e2117a68947'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:11.242500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] The token cache is being invalidate because project 7b4298aa065642c0bcfaed146905ce85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:48:11.243126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:11.244156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aa284908-4cb9-4465-a913-8e2117a68947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:11.246545 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1197/2384] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:48:11 2022] DELETE /identity/v3/projects/7b4298aa065642c0bcfaed146905ce85 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:48:20.862462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:21.239386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:54.746133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:54.747779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:54.771441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed284-57dc-4271-8479-ea3559e1513d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:54.865807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:54.866795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:54.867158 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:54.867514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] PATH_INFO: `/v3/users/afe536ef1c804099bfec9a075bd42392` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:54.873836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe536ef1c804099bfec9a075bd42392)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:54.877525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:54.967208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe536ef1c804099bfec9a075bd42392'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:54.992174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe536ef1c804099bfec9a075bd42392'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:54.993155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe536ef1c804099bfec9a075bd42392'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.009202 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe536ef1c804099bfec9a075bd42392'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.010388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe536ef1c804099bfec9a075bd42392'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.046388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1188/2385] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:48:54 2022] DELETE /identity/v3/users/afe536ef1c804099bfec9a075bd42392 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:48:55.050886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:55.051651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:55.051651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:55.071709 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:55.073181 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:55.156114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1198/2386] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:55.169304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:55.171543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:55.272994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552d197-0edf-48c3-897f-3930484bc875 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:55.280622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9a5e3-a2ab-492e-bf35-b85799a2c01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:55.281804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a5e3-a2ab-492e-bf35-b85799a2c01c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:55.282375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a5e3-a2ab-492e-bf35-b85799a2c01c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:55.282996 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a5e3-a2ab-492e-bf35-b85799a2c01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:55.302778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9a5e3-a2ab-492e-bf35-b85799a2c01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:55.306888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9a5e3-a2ab-492e-bf35-b85799a2c01c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:55.339629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1189/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:48:55.766601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:55.768388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:55.795168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464af0a4-a239-4a9e-9c25-3d8ab2a5d631 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:55.800209 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:55.801159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:55.801452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:55.801738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] PATH_INFO: `/v3/projects/990a5207faa8471dba30671255d1ebfa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:55.803054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990a5207faa8471dba30671255d1ebfa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:55.805087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:55.867073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990a5207faa8471dba30671255d1ebfa', 'request_id': 'req-c5b97a72-7a46-45f6-b129-52c374f91d19'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.875250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990a5207faa8471dba30671255d1ebfa', 'request_id': 'req-c5b97a72-7a46-45f6-b129-52c374f91d19'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.882732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990a5207faa8471dba30671255d1ebfa', 'request_id': 'req-c5b97a72-7a46-45f6-b129-52c374f91d19'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.894855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] The token cache is being invalidate because project 990a5207faa8471dba30671255d1ebfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:48:55.895154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.895844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5b97a72-7a46-45f6-b129-52c374f91d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:48:55.897480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1199/2388] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:48:55 2022] DELETE /identity/v3/projects/990a5207faa8471dba30671255d1ebfa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:48:55.905197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3744-8c14-4650-b55b-801f849b374c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:55.906014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3744-8c14-4650-b55b-801f849b374c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:55.906014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3744-8c14-4650-b55b-801f849b374c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:55.926510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-254a3744-8c14-4650-b55b-801f849b374c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:55.927854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-254a3744-8c14-4650-b55b-801f849b374c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:55.948046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:56.001763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1190/2389] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:56.007920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.008206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.008530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.029946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:56.031191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:56.039478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1200/2390] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:48:56.044667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3744-8c14-4650-b55b-801f849b374c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.054985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3744-8c14-4650-b55b-801f849b374c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.060971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.061670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.061952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.062237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.062980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:48:56.063287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.064974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.078509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1191/2391] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:48:56.083189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.093464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fa1ce-9992-43d4-96ea-476e5c55f119 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.098510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.099250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.099535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.099856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.100668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.102789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.133409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1201/2392] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:48:56 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:48:56.138121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.149326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34098a5-8ec6-4404-917c-bb8b8c555e89 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.216190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.216930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.217217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.217525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.218227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.220236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.245253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1192/2393] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:48:56 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:48:56.249925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.260861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57be0f8-1be8-4c6a-b30c-254bdf319518 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.266232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.266928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.267211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.267488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.268257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.269959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.285696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1202/2394] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:48:56.289834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.301161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3c0bc-e6c9-4e5b-99c0-c8ffde839aba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.306617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d79075-a99f-4520-b206-e60d06af4d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.307310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d79075-a99f-4520-b206-e60d06af4d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.307715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d79075-a99f-4520-b206-e60d06af4d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.307997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d79075-a99f-4520-b206-e60d06af4d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.308773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d79075-a99f-4520-b206-e60d06af4d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.310608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d79075-a99f-4520-b206-e60d06af4d50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.322188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1193/2395] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:48:56.326958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.337697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb9fc-54c6-47ff-a210-fdc036d3fec5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.343636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.344590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.344874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.345152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] PATH_INFO: `/v3/projects/682dc79f350e4c8b99273e547bbcbb65/users/902939b42657460ba6d7496fa7b9b5e6/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.363527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=682dc79f350e4c8b99273e547bbcbb65, user_id=902939b42657460ba6d7496fa7b9b5e6, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.366305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.385314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1203/2396] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:48:56 2022] PUT /identity/v3/projects/682dc79f350e4c8b99273e547bbcbb65/users/902939b42657460ba6d7496fa7b9b5e6/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:48:56.396546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e34c8-443b-43b8-8553-10d04f17f9f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.396845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e34c8-443b-43b8-8553-10d04f17f9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.397129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e34c8-443b-43b8-8553-10d04f17f9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.443511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2b9e34c8-443b-43b8-8553-10d04f17f9f2 None None] MFA Rules not processed for user `902939b42657460ba6d7496fa7b9b5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:48:56.445525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2b9e34c8-443b-43b8-8553-10d04f17f9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:56.550942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1194/2397] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:48:56.562575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:48:56.564443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:48:56.597079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:48:56.659819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903fbaf-ec1c-42fa-bab1-74d5dc28f52a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:48:56.664521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:48:56.665221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:48:56.665535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:48:56.665820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:48:56.683518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:48:56.686094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:48:56.704379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1204/2398] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:48:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:49:04.052632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:04.053088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:04.053203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:04.072281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:04.073567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:04.088233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:04.142305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1195/2399] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:04.203011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:04.205402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:04.297129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc026-e10f-462d-914d-5406af94cd6a service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:04.302559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61adf9-38ce-4157-92b2-ce3d723b3233 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:04.303218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61adf9-38ce-4157-92b2-ce3d723b3233 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:04.303496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61adf9-38ce-4157-92b2-ce3d723b3233 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:04.303774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61adf9-38ce-4157-92b2-ce3d723b3233 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:04.312991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61adf9-38ce-4157-92b2-ce3d723b3233 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:04.314627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61adf9-38ce-4157-92b2-ce3d723b3233 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:04.323292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1205/2400] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:49:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:49:05.272535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:05.274339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:05.381856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca491e52-fdeb-47f7-9481-09896ae8db56 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:05.389521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:05.390862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:05.391367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:05.391815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] PATH_INFO: `/v3/users/a88a4c921cd74128afcd6c23342d8d15` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:05.393898 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88a4c921cd74128afcd6c23342d8d15)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:05.397311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:05.465317 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88a4c921cd74128afcd6c23342d8d15'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:05.483206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88a4c921cd74128afcd6c23342d8d15'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:05.484088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88a4c921cd74128afcd6c23342d8d15'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:05.493035 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88a4c921cd74128afcd6c23342d8d15'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:05.493862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88a4c921cd74128afcd6c23342d8d15'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:05.508393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1196/2401] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:49:05 2022] DELETE /identity/v3/users/a88a4c921cd74128afcd6c23342d8d15 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:49:05.512936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:05.513847 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:05.514132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:05.533360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:05.534710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:05.617059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1206/2402] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:05.629003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:05.631117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:05.700597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:05.719046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb8902-c15e-47d9-95fa-323ea151c93e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:05.725665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:05.726554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:05.727025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:05.727308 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:05.740049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:05.742127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:05.756982 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1197/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:06.143944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.144228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.155142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:06.155398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:06.167677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15699989-8a64-44c0-903f-5be3865bebb9 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.181440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9105-a7fb-4c3e-a9b2-72b98a85b9a0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.184688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.185798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.186093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.186379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] PATH_INFO: `/v3/users/902939b42657460ba6d7496fa7b9b5e6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.187621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902939b42657460ba6d7496fa7b9b5e6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.189771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.203398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.204277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.204650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.204997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] PATH_INFO: `/v3/projects/1b57613abfd645d687bfe07afd8b3d32` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.212605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b57613abfd645d687bfe07afd8b3d32)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.215178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.304378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:06.306786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b57613abfd645d687bfe07afd8b3d32', 'request_id': 'req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.309011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902939b42657460ba6d7496fa7b9b5e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.329173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b57613abfd645d687bfe07afd8b3d32', 'request_id': 'req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.347372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b57613abfd645d687bfe07afd8b3d32', 'request_id': 'req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.351773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902939b42657460ba6d7496fa7b9b5e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.353063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902939b42657460ba6d7496fa7b9b5e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.371951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] The token cache is being invalidate because project 1b57613abfd645d687bfe07afd8b3d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:49:06.372468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902939b42657460ba6d7496fa7b9b5e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.373001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.373943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-befa988b-3f4e-47dd-bdfd-3d7d3780ad54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.374254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8056f55a-c807-43b4-a1b6-c568ea1c2fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902939b42657460ba6d7496fa7b9b5e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:06.376249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1207/2404] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:49:06 2022] DELETE /identity/v3/projects/1b57613abfd645d687bfe07afd8b3d32 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:49:06.383146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73bda8-0eeb-42db-bbdd-59941e25426a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.383628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73bda8-0eeb-42db-bbdd-59941e25426a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.384018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73bda8-0eeb-42db-bbdd-59941e25426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.413720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1198/2405] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:49:06 2022] DELETE /identity/v3/users/902939b42657460ba6d7496fa7b9b5e6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:49:06.416353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.416648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.416930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.427020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6a73bda8-0eeb-42db-bbdd-59941e25426a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:06.429144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a73bda8-0eeb-42db-bbdd-59941e25426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:06.451559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:06.452934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:06.562887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1208/2406] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:06.574503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.575702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.576219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.600726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1199/2407] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:06.603019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:06.610046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:06.612252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.615056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:06.622588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1209/2408] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:06.626353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.639964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761e52a-f04f-48e5-a599-9bf90ba8583c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.652061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.653717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.654010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.654337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.655080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:49:06.655394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.657104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.673233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1210/2409] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:49:06.677242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.689509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d11270-74db-4c77-8c0a-a425bfd5bc69 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.694529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.695528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.695856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.696559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.697515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.699582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.712648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9b72b-85bf-472d-9333-0f9d449d23f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.718377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.719390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.719910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.720538 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.735557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.738551 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.745731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1211/2410] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:49:06 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:49:06.747872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.750034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:06.760837 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1200/2411] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:06.848005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812dc15-27e5-4a5e-8013-36ca038fabe4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.859025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.859788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.860774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.861256 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.862184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.864634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.899306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1212/2412] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:49:06 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:49:06.900636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.913074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8262cf2-8ae5-423e-8843-f7c9c25c7c89 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:06.947765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:06.948672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:06.949036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:06.949460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:06.950994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:06.953064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:06.966565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1201/2413] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:49:06.970696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:06.993226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242cfc8-f74f-4fc7-9fb2-a32e896d008c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:07.003312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63f79d-ab2e-4e73-80b7-78b931581c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:07.004536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63f79d-ab2e-4e73-80b7-78b931581c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:07.005052 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63f79d-ab2e-4e73-80b7-78b931581c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:07.005639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63f79d-ab2e-4e73-80b7-78b931581c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:07.007255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63f79d-ab2e-4e73-80b7-78b931581c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:07.010711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63f79d-ab2e-4e73-80b7-78b931581c16 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:07.027797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1213/2414] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:49:07.032149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:07.045025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238f954-2026-4583-a74b-b8dcb2b805c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:07.050654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:07.053400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:07.053400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:07.053400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] PATH_INFO: `/v3/projects/b9cd16eebd78479b9467785a11afa422/users/dff0938c8cbf4ed09d143455f07fea74/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:07.072952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cd16eebd78479b9467785a11afa422, user_id=dff0938c8cbf4ed09d143455f07fea74, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:07.077914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:07.097530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1202/2415] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:49:07 2022] PUT /identity/v3/projects/b9cd16eebd78479b9467785a11afa422/users/dff0938c8cbf4ed09d143455f07fea74/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:49:07.109365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:07.109905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:07.110358 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:07.145913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:07.151644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] MFA Rules not processed for user `dff0938c8cbf4ed09d143455f07fea74`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:07.153435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:07.168762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:07.170820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:07.273856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcc215-750e-43b0-acea-d4e8963c0abf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:07.281845 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:07.283076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:07.283531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:07.284177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] PATH_INFO: `/v3/projects/682dc79f350e4c8b99273e547bbcbb65` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:07.286362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] RBAC: Authorizing `identity:delete_project(project_id=682dc79f350e4c8b99273e547bbcbb65)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:07.288572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:07.302030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1214/2416] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:49:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:07.312472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:07.314530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:07.358408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '682dc79f350e4c8b99273e547bbcbb65', 'request_id': 'req-ee84e594-0594-47fa-962d-653afbd6696d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.365595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '682dc79f350e4c8b99273e547bbcbb65', 'request_id': 'req-ee84e594-0594-47fa-962d-653afbd6696d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.380029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '682dc79f350e4c8b99273e547bbcbb65', 'request_id': 'req-ee84e594-0594-47fa-962d-653afbd6696d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.394218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] The token cache is being invalidate because project 682dc79f350e4c8b99273e547bbcbb65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:49:07.394598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.395887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.397849 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1203/2417] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:49:07 2022] DELETE /identity/v3/projects/682dc79f350e4c8b99273e547bbcbb65 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:49:07.441572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab174a2-1353-4ce7-9120-08e021268131 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:07.559357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:07.560144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:07.560536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:07.560983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:07.565919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:07.713904 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:07.715954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcf6ad-3999-4e4b-9ba9-953fdd60a986 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:07.750282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1215/2418] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:49:07.790711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:07.792561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:07.870286 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:07.878353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84e594-0594-47fa-962d-653afbd6696d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:07.884772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:07.886166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:07.886689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:07.887212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] PATH_INFO: `/v3/users/dff0938c8cbf4ed09d143455f07fea74` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:07.888868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff0938c8cbf4ed09d143455f07fea74)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:07.891038 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:07.965913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff0938c8cbf4ed09d143455f07fea74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.983253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff0938c8cbf4ed09d143455f07fea74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.984020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff0938c8cbf4ed09d143455f07fea74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.993945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff0938c8cbf4ed09d143455f07fea74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:07.995298 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff0938c8cbf4ed09d143455f07fea74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:08.015075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1204/2419] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:49:07 2022] DELETE /identity/v3/users/dff0938c8cbf4ed09d143455f07fea74 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:49:08.020472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:08.021111 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:08.021660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:08.046144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:08.048131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:08.057998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:08.133384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1216/2420] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:08.146662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:08.148812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:08.221687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbba51e-b580-42b7-95d3-761442949da4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:08.228900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4f52c-5342-4d25-abe5-e673963ee010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:08.229586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f52c-5342-4d25-abe5-e673963ee010 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:08.229870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f52c-5342-4d25-abe5-e673963ee010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:08.230157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f52c-5342-4d25-abe5-e673963ee010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:08.241297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4f52c-5342-4d25-abe5-e673963ee010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:08.242869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4f52c-5342-4d25-abe5-e673963ee010 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:08.255189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1205/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:08.623853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:08.625575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:08.649579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eebdb5-81f4-44cc-a5e0-71afc327d981 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:08.660935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:08.662056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:08.662346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:08.662624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] PATH_INFO: `/v3/projects/b9cd16eebd78479b9467785a11afa422` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:08.663896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9cd16eebd78479b9467785a11afa422)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:08.665996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:08.755815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9cd16eebd78479b9467785a11afa422', 'request_id': 'req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:08.767580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9cd16eebd78479b9467785a11afa422', 'request_id': 'req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:08.782165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9cd16eebd78479b9467785a11afa422', 'request_id': 'req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:08.823386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] The token cache is being invalidate because project b9cd16eebd78479b9467785a11afa422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:49:08.825194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:08.826889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-702d5364-ea62-4a32-9e5a-6d8c69bf1eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:49:08.833229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1217/2422] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:49:08 2022] DELETE /identity/v3/projects/b9cd16eebd78479b9467785a11afa422 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:49:18.250070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:18.815923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:19.291719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec2f4d-d115-40b6-bfa6-2ef663a84426 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.292330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec2f4d-d115-40b6-bfa6-2ef663a84426 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.293063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec2f4d-d115-40b6-bfa6-2ef663a84426 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.294914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1206/2423] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:19.300928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7b91-0894-44f0-aff7-791bf087a411 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.301224 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7b91-0894-44f0-aff7-791bf087a411 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.301509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7b91-0894-44f0-aff7-791bf087a411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.353396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bbbb7b91-0894-44f0-aff7-791bf087a411 None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:19.354533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bbbb7b91-0894-44f0-aff7-791bf087a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:19.443789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1218/2424] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:19.456549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec2f4d-d115-40b6-bfa6-2ef663a84426 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:19.458260 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ebec2f4d-d115-40b6-bfa6-2ef663a84426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:19.519009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec2f4d-d115-40b6-bfa6-2ef663a84426 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:19.522462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a36da-e9ed-45da-ace4-c2b64e7ac123 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:19.523106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a36da-e9ed-45da-ace4-c2b64e7ac123 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.523387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a36da-e9ed-45da-ace4-c2b64e7ac123 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.523665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a36da-e9ed-45da-ace4-c2b64e7ac123 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.534578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a36da-e9ed-45da-ace4-c2b64e7ac123 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:19.536016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a36da-e9ed-45da-ace4-c2b64e7ac123 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:19.547479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1207/2425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:19.615320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57b084-3e6e-43c8-8eb3-1bac5ff67d74 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.615616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57b084-3e6e-43c8-8eb3-1bac5ff67d74 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.615898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57b084-3e6e-43c8-8eb3-1bac5ff67d74 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.617316 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1219/2426] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:19.622102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300da01-b5c2-47cc-8685-a6ee9292e2c9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.622403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300da01-b5c2-47cc-8685-a6ee9292e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.622676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300da01-b5c2-47cc-8685-a6ee9292e2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.642605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c300da01-b5c2-47cc-8685-a6ee9292e2c9 None None] MFA Rules not processed for user `6f44d10be1654d17990966ae64d43ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:19.644618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c300da01-b5c2-47cc-8685-a6ee9292e2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:19.698874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1208/2427] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:19.710850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57b084-3e6e-43c8-8eb3-1bac5ff67d74 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:19.712605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0f57b084-3e6e-43c8-8eb3-1bac5ff67d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:19.726363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57b084-3e6e-43c8-8eb3-1bac5ff67d74 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:19.729917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a34831-a5f2-4b62-a11d-58a9ed509c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:19.731235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a34831-a5f2-4b62-a11d-58a9ed509c5c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.731535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a34831-a5f2-4b62-a11d-58a9ed509c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.731800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a34831-a5f2-4b62-a11d-58a9ed509c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.738985 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a34831-a5f2-4b62-a11d-58a9ed509c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:19.740470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a34831-a5f2-4b62-a11d-58a9ed509c5c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:19.748257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1220/2428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:19.831398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4f7e7-7672-4974-b44c-306a87bc042b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.831691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4f7e7-7672-4974-b44c-306a87bc042b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.831973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4f7e7-7672-4974-b44c-306a87bc042b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.833653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1209/2429] 10.222.0.16 () {54 vars in 921 bytes} [Thu May 26 13:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 13:49:19.839173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bcc6e-6f2e-41d3-b8b4-3f5c69127aea None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.840211 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bcc6e-6f2e-41d3-b8b4-3f5c69127aea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.840477 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bcc6e-6f2e-41d3-b8b4-3f5c69127aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.859205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-164bcc6e-6f2e-41d3-b8b4-3f5c69127aea None None] MFA Rules not processed for user `535f23db24704422a4128ff09f1cc284`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:19.860442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-164bcc6e-6f2e-41d3-b8b4-3f5c69127aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:19.869723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1221/2430] 10.222.0.16 () {58 vars in 1018 bytes} [Thu May 26 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:19.880323 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4f7e7-7672-4974-b44c-306a87bc042b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:19.892067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4f7e7-7672-4974-b44c-306a87bc042b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:19.895687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:19.896351 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:19.896749 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:19.897039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:19.908045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:19.909985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:19.922398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1210/2431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:22.437203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:22.437694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:22.437788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:22.456834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:22.458079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:22.514153 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1222/2432] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:49:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:49:22.525999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:22.527532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:22.602986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b25bb-38a6-4bfb-9435-9da370a9ed44 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:22.608060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:22.608741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:22.609054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:22.609335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:22.617485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:22.619100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:22.627903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1211/2433] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:49:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:49:32.506278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=80 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:32.624660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:41.872066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:41.873809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:41.885161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10d405-54d0-417e-a003-d28e26f629bf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:41.889352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:41.890006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:41.890291 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:41.890571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:41.892263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:41.905546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:41.906986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:41.919943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1223/2434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:43.097236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:43.107043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac96419-5ef4-4925-9d67-5594042cd126 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:43.110667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:43.111293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:43.111572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:43.111852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:43.113611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:43.127831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:43.129314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc431c2-112f-41cf-bb2b-251ff7895878 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:43.141080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1212/2435] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:44.273808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:44.275223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:44.289382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8bc9e-f8ca-4e8a-8ea6-50d2befc4226 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:44.295452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:44.296109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:44.296427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:44.296708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:44.298529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:44.359742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:44.361384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:44.371653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1224/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:49:53.136121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:54.366579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=34 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:49:55.418328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:55.418851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:55.418958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:55.436897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:49:55.438104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:55.445770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1213/2437] 10.222.0.16 () {58 vars in 967 bytes} [Thu May 26 13:49:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:49:55.458637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:55.470296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1b819-eaa0-4e7a-8843-8d9fb8a4a136 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:55.475611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:55.476257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:55.476575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:55.476894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:55.486368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:55.488092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:55.496389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1225/2438] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:49:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:49:59.482069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:49:59.492495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f470-8cb1-45e6-8ba4-3d8e9b6407bc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:49:59.497429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:49:59.498550 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:49:59.499065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:49:59.499769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:49:59.502952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:49:59.518486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:49:59.520201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:49:59.530020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1214/2439] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:49:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:05.493078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:09.525618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:10.402714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:10.404900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:10.423936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b188fd-066d-46d6-b437-7b16c1ba4eed service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:10.429525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:10.430177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:10.430466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:10.431247 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:10.433300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:10.492008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:10.494288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a66fa8-2699-4b04-baae-0d908d943f39 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:10.503609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1226/2440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:50:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:20.499160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=28 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:23.819031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:23.832725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc3c85-be78-4c72-b3b1-9f8d802a0b36 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:23.838582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:23.839253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:23.839546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:23.839829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:23.841690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:23.904917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:23.906487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:23.917408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1215/2441] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:50:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:32.929460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d62f1-94ab-4292-91e0-e36be2daa671 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:32.930549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d62f1-94ab-4292-91e0-e36be2daa671 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:32.930761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d62f1-94ab-4292-91e0-e36be2daa671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:32.948988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f46d62f1-94ab-4292-91e0-e36be2daa671 None None] MFA Rules not processed for user `dd8055e46e6149d3897fb88af0b19d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:50:32.950185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f46d62f1-94ab-4292-91e0-e36be2daa671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:32.957267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1227/2442] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:50:33.911697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:39.415353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:50:39.430582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c02fa6-e2d6-48ee-9ebe-786ccec369ed service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:50:39.443174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:50:39.444634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:50:39.446039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:50:39.446562 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:50:39.449347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:50:39.468057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:50:39.470439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:50:39.487493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1216/2443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:50:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:50:42.943077 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:50:49.480605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:51:11.397061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:11.397631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:11.397631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:11.416312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] MFA Rules not processed for user `e13f5cad16824554a1fb32ed9a915ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:51:11.417457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:11.424274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1228/2444] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:51:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:51:11.734113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:11.744502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aea31-b539-440b-9ff6-d5cbb7694d7b service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:11.748224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:11.748916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:11.749205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:11.749485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:11.751330 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:11.761729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:11.763217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:11.772499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1217/2445] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:51:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:51:13.099196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:13.103381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:13.192819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd998-0678-4357-aba6-62ce765cea4b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:13.200911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:13.201574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:13.201857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:13.202519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:13.203908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:13.217073 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:13.219047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906f65e-2e0d-4ab1-b5f4-32e5c2a55197 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:13.231622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1229/2446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:51:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:51:15.503234 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:15.521506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8778bb-c412-4be0-a22c-fcf0d1612f7a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:15.528389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:15.529509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:15.529954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:15.530528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:15.533009 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:15.554863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:15.557810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:15.572320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1218/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:51:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:51:23.226618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=35 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:51:25.566643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:51:31.119814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:31.120282 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:31.120421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:31.138335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:51:31.139553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:31.147587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1230/2448] 10.222.0.16 () {58 vars in 967 bytes} [Thu May 26 13:51:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:51:31.200452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:31.212549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04be3a-56f0-493f-a5ad-b82d8b82f441 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:31.217832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:31.218474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:31.218784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:31.219059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:31.227549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:31.229179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:31.237092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1219/2449] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:51:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 13:51:31.590844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:31.600127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bd773-c698-4efc-9b4e-0655e63a2fd0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:31.603689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:31.604362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:31.604651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:31.604931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:31.606708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:31.669966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:31.671414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:31.681034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1231/2450] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:51:34.889773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:34.903465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce6b68-c6c5-4d37-9f88-b307ce5021d8 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:34.909532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:34.912248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:34.912577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:34.912865 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:34.915633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:34.938225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:34.940232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:34.958386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1220/2451] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:51:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:51:38.214592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:38.216034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:38.236482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4adba-faed-4f08-9638-3a9610353a2d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:38.242972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:38.243869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:38.244155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:38.244469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] PATH_INFO: `/v3/users/5b0a34631db94d1d9fd5e865de5b9156` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:38.245688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0a34631db94d1d9fd5e865de5b9156)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:38.247639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:38.300787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0a34631db94d1d9fd5e865de5b9156'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.302043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0a34631db94d1d9fd5e865de5b9156'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.309368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0a34631db94d1d9fd5e865de5b9156'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.310076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0a34631db94d1d9fd5e865de5b9156'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.319105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0a34631db94d1d9fd5e865de5b9156'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.339434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1232/2452] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:51:38 2022] DELETE /identity/v3/users/5b0a34631db94d1d9fd5e865de5b9156 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:51:38.520566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:38.531756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c59b8-5c3a-4bae-8ce5-385f91df8cb0 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:38.592855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:38.593739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:38.594028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:38.594314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] PATH_INFO: `/v3/projects/c70c0ab9b5134d0781dea0e0abc02ac8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:38.595546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70c0ab9b5134d0781dea0e0abc02ac8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:38.597540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:38.649393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70c0ab9b5134d0781dea0e0abc02ac8', 'request_id': 'req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.655394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70c0ab9b5134d0781dea0e0abc02ac8', 'request_id': 'req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.667106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70c0ab9b5134d0781dea0e0abc02ac8', 'request_id': 'req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.679866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] The token cache is being invalidate because project c70c0ab9b5134d0781dea0e0abc02ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:51:38.680160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.680879 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.682496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1221/2453] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:51:38 2022] DELETE /identity/v3/projects/c70c0ab9b5134d0781dea0e0abc02ac8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:51:38.686673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:38.688183 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:38.701849 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9259d-a3f9-4235-9e87-226eecb30ee5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:38.771246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:38.772750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:38.773271 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:38.773785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] PATH_INFO: `/v3/users/f1eb82f3cf544e878ff395ebbd34a85a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:38.775824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1eb82f3cf544e878ff395ebbd34a85a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:38.779432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:38.781695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:51:38.845693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1eb82f3cf544e878ff395ebbd34a85a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.846395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1eb82f3cf544e878ff395ebbd34a85a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.856803 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1eb82f3cf544e878ff395ebbd34a85a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.857555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1eb82f3cf544e878ff395ebbd34a85a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.867638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1eb82f3cf544e878ff395ebbd34a85a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:38.888919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1233/2454] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:51:38 2022] DELETE /identity/v3/users/f1eb82f3cf544e878ff395ebbd34a85a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:51:39.100862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:51:39.102905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:51:39.117875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89eadc-d49b-43b4-9c5e-b83ff26b7527 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:51:39.188519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:51:39.189389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:51:39.189669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:51:39.189944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] PATH_INFO: `/v3/projects/25d42e3d3a094377b459e21f70fd3455` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:51:39.191184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d42e3d3a094377b459e21f70fd3455)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:51:39.193174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:51:39.200361 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:51:39.248404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d42e3d3a094377b459e21f70fd3455', 'request_id': 'req-6e4898ff-fe80-46c5-bde0-019738053619'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:39.254452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d42e3d3a094377b459e21f70fd3455', 'request_id': 'req-6e4898ff-fe80-46c5-bde0-019738053619'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:39.266958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d42e3d3a094377b459e21f70fd3455', 'request_id': 'req-6e4898ff-fe80-46c5-bde0-019738053619'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:39.281341 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] The token cache is being invalidate because project 25d42e3d3a094377b459e21f70fd3455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:51:39.281634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:39.282321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6e4898ff-fe80-46c5-bde0-019738053619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:51:39.283931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1222/2455] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:51:39 2022] DELETE /identity/v3/projects/25d42e3d3a094377b459e21f70fd3455 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:51:48.884772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:51:49.277988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:01.190255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:01.192697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:01.288956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efe9e0-3680-4358-a5b4-51b785e2c949 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:01.350138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:01.350905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:01.351378 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:01.351850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:01.354012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:01.427798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:01.429338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:01.439622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1234/2456] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:52:02.515531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:02.516092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:02.516226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:02.535921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] MFA Rules not processed for user `9958aaa8690c468da2acec47b2bb5f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:02.537268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:02.605012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1223/2457] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:05.485411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:05.488652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:05.515198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a57531-0975-41c0-a832-739d97c959b9 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:05.522104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:05.523334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:05.523653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:05.523964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] PATH_INFO: `/v3/users/dd8055e46e6149d3897fb88af0b19d39` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:05.525296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8055e46e6149d3897fb88af0b19d39)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:05.527497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:05.585673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8055e46e6149d3897fb88af0b19d39'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:05.586438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8055e46e6149d3897fb88af0b19d39'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:05.594182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8055e46e6149d3897fb88af0b19d39'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:05.595025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8055e46e6149d3897fb88af0b19d39'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:05.599975 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:05.606557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-94f0e90e-f992-4666-8113-3785a6abf321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8055e46e6149d3897fb88af0b19d39'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:05.629409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1235/2458] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:05 2022] DELETE /identity/v3/users/dd8055e46e6149d3897fb88af0b19d39 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:06.053528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.066617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60ed25-57e1-438b-bd21-1d9dc2c07bf4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.150435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.151608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.153164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.153164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] PATH_INFO: `/v3/projects/667a2ccd9c9045879b4b87e156997161` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.153820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] RBAC: Authorizing `identity:delete_project(project_id=667a2ccd9c9045879b4b87e156997161)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.156474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.225607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667a2ccd9c9045879b4b87e156997161', 'request_id': 'req-2905ded2-cce5-4c65-b627-5901503d710a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:06.232113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667a2ccd9c9045879b4b87e156997161', 'request_id': 'req-2905ded2-cce5-4c65-b627-5901503d710a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:06.242333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667a2ccd9c9045879b4b87e156997161', 'request_id': 'req-2905ded2-cce5-4c65-b627-5901503d710a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:06.258783 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] The token cache is being invalidate because project 667a2ccd9c9045879b4b87e156997161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:06.259084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:06.259755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2905ded2-cce5-4c65-b627-5901503d710a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:06.261384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1224/2459] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:06 2022] DELETE /identity/v3/projects/667a2ccd9c9045879b4b87e156997161 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:06.266548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.266783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.267190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.296363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:06.297798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:06.390679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1236/2460] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:06.399684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.400197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.400732 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.426185 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:06.427410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:06.445240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1225/2461] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:06.450778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.466257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebd1ee-08a3-4a5b-869e-b7d087d429dc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.474527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.475777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.476257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.476704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.477529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:52:06.477962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.480430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.510662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.516407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1237/2462] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:52:06.522623 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4902f6-7a93-4ba0-9320-8c2adb1924c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.529542 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.530075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.530362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.530647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.531369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.544027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.601339 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:06.612204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1226/2463] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:06 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:52:06.614333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.625752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a8867-b2b0-41fa-8599-d03b9a26fc88 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.688898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.689606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.689879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.690157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.690902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.692903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.718850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1238/2464] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:06 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:52:06.720104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.731295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8c4bb-c2f5-4e9b-94a8-233b64901d62 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.738746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778872-9394-4623-a884-24358f807247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.740100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778872-9394-4623-a884-24358f807247 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.740491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778872-9394-4623-a884-24358f807247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.741532 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778872-9394-4623-a884-24358f807247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.742207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e778872-9394-4623-a884-24358f807247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.745296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e778872-9394-4623-a884-24358f807247 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.760152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1227/2465] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:06.761419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.774051 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a04a4-7c49-4311-a6b3-cbf2e95fe09d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.779147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb864c4d-b2a7-496b-8fce-ba773222c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.780118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864c4d-b2a7-496b-8fce-ba773222c088 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.780118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864c4d-b2a7-496b-8fce-ba773222c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.780393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864c4d-b2a7-496b-8fce-ba773222c088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.781498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb864c4d-b2a7-496b-8fce-ba773222c088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.783004 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb864c4d-b2a7-496b-8fce-ba773222c088 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.802488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1239/2466] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:06.803692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778872-9394-4623-a884-24358f807247 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:06.815337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778872-9394-4623-a884-24358f807247 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:06.820666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:06.821948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.822233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.822514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] PATH_INFO: `/v3/projects/36c9e727df044de4a205a8454716bdd4/users/1b4b950267154641884aed5f8ca6b2d6/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.847764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c9e727df044de4a205a8454716bdd4, user_id=1b4b950267154641884aed5f8ca6b2d6, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:06.851229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:06.870985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1228/2467] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:06 2022] PUT /identity/v3/projects/36c9e727df044de4a205a8454716bdd4/users/1b4b950267154641884aed5f8ca6b2d6/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:06.888831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:06.889159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:06.889395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:06.925881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] MFA Rules not processed for user `1b4b950267154641884aed5f8ca6b2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:06.927239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:06.978785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:07.021702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1240/2468] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:07.026348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.089946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0ba6-5417-453e-bed6-f92540d6a912 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.094984 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.095662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.095944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.096218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.097629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.099739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.126641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1229/2469] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:07 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:07.130115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.141129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f7c47-dd96-42a7-aca1-70401ccbb3b3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.201148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.201635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.201820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.202103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.202799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.204813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.226887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1241/2470] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:07 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:07.230801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.241670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b02abe-fb7c-4a72-8a42-3d361812815c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.246639 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.247316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.247604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.247886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.248684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.250296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.262002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1230/2471] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:07.266236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.277234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b438e-5cec-4d7d-8a89-fb4e6254205b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.282022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.282902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.283210 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.283491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] PATH_INFO: `/v3/projects/8d934e991789477ab33c75cee41d219e/users/3d9a4bb0f13c4e69ac962e0ab8fa7311/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.303462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d934e991789477ab33c75cee41d219e, user_id=3d9a4bb0f13c4e69ac962e0ab8fa7311, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.306161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.321200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1242/2472] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:07 2022] PUT /identity/v3/projects/8d934e991789477ab33c75cee41d219e/users/3d9a4bb0f13c4e69ac962e0ab8fa7311/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:07.325221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.335993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e3483-63e8-485b-8e58-83e20fc610fa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.402759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcac3e-9662-4a80-a154-684dbac442b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.403437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcac3e-9662-4a80-a154-684dbac442b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.403728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcac3e-9662-4a80-a154-684dbac442b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.403992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcac3e-9662-4a80-a154-684dbac442b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.404760 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcac3e-9662-4a80-a154-684dbac442b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.406387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcac3e-9662-4a80-a154-684dbac442b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.417016 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1231/2473] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:07.421427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.431993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7257c-e29e-4f8d-a862-f0250dd85ce1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.436701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.437591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.437880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.438164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] PATH_INFO: `/v3/projects/8d934e991789477ab33c75cee41d219e/users/3d9a4bb0f13c4e69ac962e0ab8fa7311/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.440157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d934e991789477ab33c75cee41d219e, user_id=3d9a4bb0f13c4e69ac962e0ab8fa7311, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.443808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.454828 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1243/2474] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:07 2022] PUT /identity/v3/projects/8d934e991789477ab33c75cee41d219e/users/3d9a4bb0f13c4e69ac962e0ab8fa7311/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:07.467622 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.468070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.468571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.497802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] MFA Rules not processed for user `3d9a4bb0f13c4e69ac962e0ab8fa7311`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:07.499008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:07.521730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:07.573736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1232/2475] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:07.585993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:07.588055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:07.662133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e690-53fd-4c8c-810e-9739bdf6aaed admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:07.666448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76a2a-43c2-4822-9c3c-74ceef4f77b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:07.667137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76a2a-43c2-4822-9c3c-74ceef4f77b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:07.667609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76a2a-43c2-4822-9c3c-74ceef4f77b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:07.668049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76a2a-43c2-4822-9c3c-74ceef4f77b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:07.678793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df76a2a-43c2-4822-9c3c-74ceef4f77b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:07.680265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df76a2a-43c2-4822-9c3c-74ceef4f77b7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:07.693942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1244/2476] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:08.119777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:08.190099 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210d3a1-4e67-4687-a8bd-5794b1285b5e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:08.196073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:08.197024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:08.197331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:08.197608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] PATH_INFO: `/v3/users/1b4b950267154641884aed5f8ca6b2d6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:08.198912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4b950267154641884aed5f8ca6b2d6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:08.200930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:08.275676 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4b950267154641884aed5f8ca6b2d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.290109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4b950267154641884aed5f8ca6b2d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.290874 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4b950267154641884aed5f8ca6b2d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.299320 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4b950267154641884aed5f8ca6b2d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.300931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4b950267154641884aed5f8ca6b2d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.315343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1233/2477] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:08 2022] DELETE /identity/v3/users/1b4b950267154641884aed5f8ca6b2d6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:08.320561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:08.320856 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:08.321197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:08.341502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:08.342732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:08.377944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:08.420296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1245/2478] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:08.430867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:08.432336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:08.477973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:08.499097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03e3c-76b8-4314-928e-cea4d5dc00db admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:08.503021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:08.503716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:08.503995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:08.504264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:08.514741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:08.516222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:08.527556 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1234/2479] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:08.855794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:08.857778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:08.872571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fb093-497d-4138-ba19-3eea68a8fbe5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:08.878464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:08.879811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:08.880164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:08.880549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] PATH_INFO: `/v3/projects/36c9e727df044de4a205a8454716bdd4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:08.881924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c9e727df044de4a205a8454716bdd4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:08.884286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:08.938326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c9e727df044de4a205a8454716bdd4', 'request_id': 'req-2d5813da-bf39-4943-ba42-66c67a68951a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.947575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c9e727df044de4a205a8454716bdd4', 'request_id': 'req-2d5813da-bf39-4943-ba42-66c67a68951a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.955045 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c9e727df044de4a205a8454716bdd4', 'request_id': 'req-2d5813da-bf39-4943-ba42-66c67a68951a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.966973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] The token cache is being invalidate because project 36c9e727df044de4a205a8454716bdd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:08.967270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.967935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:08.969535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1246/2480] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:08 2022] DELETE /identity/v3/projects/36c9e727df044de4a205a8454716bdd4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:08.974110 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:08.985904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68cd42-b7d5-403a-8c34-1172468eb8d7 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:09.060375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:09.061302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:09.061572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:09.061820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] PATH_INFO: `/v3/users/3d9a4bb0f13c4e69ac962e0ab8fa7311` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:09.063089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9a4bb0f13c4e69ac962e0ab8fa7311)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:09.065098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:09.120819 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9a4bb0f13c4e69ac962e0ab8fa7311'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.134978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9a4bb0f13c4e69ac962e0ab8fa7311'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.135738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9a4bb0f13c4e69ac962e0ab8fa7311'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.146368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9a4bb0f13c4e69ac962e0ab8fa7311'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.148531 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-37f4cde3-2141-4e32-b1f4-34bfe520d74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9a4bb0f13c4e69ac962e0ab8fa7311'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.167086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1235/2481] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:08 2022] DELETE /identity/v3/users/3d9a4bb0f13c4e69ac962e0ab8fa7311 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:09.575327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:09.577009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:09.591907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5813da-bf39-4943-ba42-66c67a68951a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:09.658998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:09.660922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:09.661429 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:09.661572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] PATH_INFO: `/v3/projects/8d934e991789477ab33c75cee41d219e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:09.663021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d934e991789477ab33c75cee41d219e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:09.665217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:09.723723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d934e991789477ab33c75cee41d219e', 'request_id': 'req-914c04a8-f7ec-48e7-a67a-e838cf6e0785'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.730293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:09.735094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d934e991789477ab33c75cee41d219e', 'request_id': 'req-914c04a8-f7ec-48e7-a67a-e838cf6e0785'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.744854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d934e991789477ab33c75cee41d219e', 'request_id': 'req-914c04a8-f7ec-48e7-a67a-e838cf6e0785'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.756919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] The token cache is being invalidate because project 8d934e991789477ab33c75cee41d219e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:09.757263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.757988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-914c04a8-f7ec-48e7-a67a-e838cf6e0785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:09.759548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1247/2482] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:09 2022] DELETE /identity/v3/projects/8d934e991789477ab33c75cee41d219e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:09.764512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:09.764886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:09.765183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:09.784844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:09.786209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:09.842261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:09.856272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1236/2483] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:09.865120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:09.865666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:09.865666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:09.884107 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:09.885326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:09.893104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1248/2484] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:09.897502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:09.907129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a8ee5-1532-47a1-817a-45662a0bfa7b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:09.912758 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:09.913424 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:09.913707 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:09.914021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:09.914716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:52:09.915019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:09.916650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:09.930138 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1237/2485] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:52:09.934537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:09.944722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a249fb7-251e-4d91-97a8-0cdee11a8932 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:09.949831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:09.950548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:09.950871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:09.951155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:09.951886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:09.953942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:09.981865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1249/2486] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:09 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:09.985403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:09.996696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e86b3-9a46-4fe7-aa13-fac8cdc917cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.054168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.054846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.055134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.055416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.056155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.058176 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.079091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1238/2487] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:09 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:10.083936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.094427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f0cd-2c5c-4af5-878a-821e25284e7b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.099745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.100452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.100779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.101056 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.101781 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.103396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.115360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1250/2488] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:10.119291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.129259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64033a-97b8-4948-a305-9c08e5653eda admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.133961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628833-ce13-4cda-ab17-faada20c7f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.134658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628833-ce13-4cda-ab17-faada20c7f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.134942 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628833-ce13-4cda-ab17-faada20c7f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.135220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628833-ce13-4cda-ab17-faada20c7f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.135935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628833-ce13-4cda-ab17-faada20c7f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.137571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628833-ce13-4cda-ab17-faada20c7f64 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.148590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1239/2489] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:10.152928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.164941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5faecd-b6ff-4d21-adb0-3742835ac0e0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.169940 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.170832 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.171114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.171391 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] PATH_INFO: `/v3/projects/4ca288d5669d4cc19231a5159c3730fb/users/77a0cb947b08411b8adfcbef8be462e5/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.188433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca288d5669d4cc19231a5159c3730fb, user_id=77a0cb947b08411b8adfcbef8be462e5, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.191453 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.206455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1251/2490] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:10 2022] PUT /identity/v3/projects/4ca288d5669d4cc19231a5159c3730fb/users/77a0cb947b08411b8adfcbef8be462e5/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:52:10.217224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.217440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.217735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.245723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] MFA Rules not processed for user `77a0cb947b08411b8adfcbef8be462e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:10.246949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:10.319335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1240/2491] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:10.323873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.400587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e2c53-00aa-4372-bbd7-72455212a4fe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.405478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.406152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.406446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.406728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.407447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.409964 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.442517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1252/2492] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:10 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:10.443971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.455063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc01ff-29c4-495d-89ac-8fe525fec43a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.506959 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:10.515679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.516402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.516706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.516986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.517736 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.520740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.552143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1241/2493] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:10 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:52:10.554544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.566245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fedba-a960-4437-b0c7-9c7ea923ea54 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.571461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.572122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.572517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.572802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.573521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.575123 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.586626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1253/2494] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:10.590701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.601315 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf5a30-7c7d-43b9-8bd1-f540f7998c91 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.605841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da789-2bee-4e47-bd43-6ee38430dd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.606489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da789-2bee-4e47-bd43-6ee38430dd8b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.606771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da789-2bee-4e47-bd43-6ee38430dd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.607050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da789-2bee-4e47-bd43-6ee38430dd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.607791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630da789-2bee-4e47-bd43-6ee38430dd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.609416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630da789-2bee-4e47-bd43-6ee38430dd8b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.619913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1242/2495] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:10.624106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.634321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb20dc-3540-43b7-90a4-79ad4fe70cc8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.638935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.639809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.640090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.640395 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] PATH_INFO: `/v3/projects/9bef378e0c1140a58c3727bd8d2a3e6e/users/d47360e6c0854d6d93d90cafe670bd79/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.654820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bef378e0c1140a58c3727bd8d2a3e6e, user_id=d47360e6c0854d6d93d90cafe670bd79, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.657559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.671357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1254/2496] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:10 2022] PUT /identity/v3/projects/9bef378e0c1140a58c3727bd8d2a3e6e/users/d47360e6c0854d6d93d90cafe670bd79/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:10.684181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.684501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.684790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.718388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] MFA Rules not processed for user `d47360e6c0854d6d93d90cafe670bd79`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:10.720350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:10.805299 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1243/2497] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:10.810484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.841972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:10.876201 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fefae-66c5-4b97-a982-f1b201b61ff1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.880971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.881635 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.881921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.882227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.882989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.885083 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:10.911886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1255/2498] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:10 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:52:10.916151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:10.928527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709ef89-cba8-49af-8cb9-334d22d2acd3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:10.986279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:10.986948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:10.987273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:10.987662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:10.988369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:10.990290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.011206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1244/2499] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:10 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:52:11.015593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:11.027119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa25a9-e658-46ec-bd08-6ed1acb33387 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:11.031920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:11.032622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.032913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.033199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.033928 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:11.036159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.047560 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1256/2500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:52:11.051988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:11.062637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349995-20df-4641-a768-bf427b09444d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:11.067764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:11.069014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.069303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.069587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] PATH_INFO: `/v3/projects/ae42a6f03c8b481a896f7137cb972fe0/users/99c28c8c1af4466ebc90fe361ec7264c/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.092096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae42a6f03c8b481a896f7137cb972fe0, user_id=99c28c8c1af4466ebc90fe361ec7264c, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:11.095085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.110248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1245/2501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:11 2022] PUT /identity/v3/projects/ae42a6f03c8b481a896f7137cb972fe0/users/99c28c8c1af4466ebc90fe361ec7264c/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:11.114337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:11.124784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a4683-83ec-4c17-bd6b-4816be578753 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:11.184907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd78a1-35e4-4e25-aca6-0a2cb6d3c028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:11.185523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd78a1-35e4-4e25-aca6-0a2cb6d3c028 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.185829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd78a1-35e4-4e25-aca6-0a2cb6d3c028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.186110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd78a1-35e4-4e25-aca6-0a2cb6d3c028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.186824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd78a1-35e4-4e25-aca6-0a2cb6d3c028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:11.188455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd78a1-35e4-4e25-aca6-0a2cb6d3c028 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.199007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1257/2502] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:11.203373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:11.215068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235f22-b343-4d3c-b1eb-e89095283cf2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:11.220207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:11.221663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.221663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.221974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] PATH_INFO: `/v3/projects/ae42a6f03c8b481a896f7137cb972fe0/users/99c28c8c1af4466ebc90fe361ec7264c/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.223802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae42a6f03c8b481a896f7137cb972fe0, user_id=99c28c8c1af4466ebc90fe361ec7264c, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:11.226721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.237938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1246/2503] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:11 2022] PUT /identity/v3/projects/ae42a6f03c8b481a896f7137cb972fe0/users/99c28c8c1af4466ebc90fe361ec7264c/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:11.248287 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.248624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.248935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.277746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] MFA Rules not processed for user `99c28c8c1af4466ebc90fe361ec7264c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:11.278995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:11.349344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1258/2504] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:11.360533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:11.362388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:11.383215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:11.423022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efb6da-8ac7-4fe9-adb4-a4af10ea8a6f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:11.426803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:11.427454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.427742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.428026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.437203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:11.438685 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.449643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1247/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:11.491067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:11.492496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:11.505134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b236d-4e1a-48e5-9459-c6189b1ce742 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:11.508794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b88c92-de60-4d3b-a8bb-1df914ddda6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:11.509440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b88c92-de60-4d3b-a8bb-1df914ddda6f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:11.509720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b88c92-de60-4d3b-a8bb-1df914ddda6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:11.509997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b88c92-de60-4d3b-a8bb-1df914ddda6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:11.530552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:11.571491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b88c92-de60-4d3b-a8bb-1df914ddda6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:11.572995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b88c92-de60-4d3b-a8bb-1df914ddda6f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:11.582143 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1259/2506] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:14.527080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:14.592477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7150b1-a0ff-4485-a215-a80d58dcc623 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:14.597374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:14.598269 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:14.598580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:14.598885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] PATH_INFO: `/v3/users/77a0cb947b08411b8adfcbef8be462e5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:14.600162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a0cb947b08411b8adfcbef8be462e5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:14.602173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:14.651950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a0cb947b08411b8adfcbef8be462e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:14.665820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a0cb947b08411b8adfcbef8be462e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:14.666547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a0cb947b08411b8adfcbef8be462e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:14.675859 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a0cb947b08411b8adfcbef8be462e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:14.676692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a0cb947b08411b8adfcbef8be462e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:14.689112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1248/2507] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:14 2022] DELETE /identity/v3/users/77a0cb947b08411b8adfcbef8be462e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:14.694009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:14.694620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:14.694620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:14.712390 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:14.713623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:14.778414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1260/2508] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:14.788774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:14.790280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:14.837687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:14.857493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f147de4-cf9d-45d5-a948-0d290c80200e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:14.861488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:14.862156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:14.862463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:14.862743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:14.872716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:14.874196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:14.885045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1249/2509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:15.227105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.229041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.261415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.262888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.282156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338033bb-f0b5-40c6-bd1c-b392fe8be20a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.287115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.288004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.288380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.288600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] PATH_INFO: `/v3/projects/4ca288d5669d4cc19231a5159c3730fb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.290530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca288d5669d4cc19231a5159c3730fb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.292808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.344998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95eebb-cb79-4fef-bd5b-3ae66d49e121 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.351398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca288d5669d4cc19231a5159c3730fb', 'request_id': 'req-08344491-b5f3-402c-8c91-7131a25c8953'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.358282 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca288d5669d4cc19231a5159c3730fb', 'request_id': 'req-08344491-b5f3-402c-8c91-7131a25c8953'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.368149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca288d5669d4cc19231a5159c3730fb', 'request_id': 'req-08344491-b5f3-402c-8c91-7131a25c8953'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.387226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] The token cache is being invalidate because project 4ca288d5669d4cc19231a5159c3730fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:15.387579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.388423 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.390559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1250/2510] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:15 2022] DELETE /identity/v3/projects/4ca288d5669d4cc19231a5159c3730fb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:15.394851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.396468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.414404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:15.431903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.432614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.432974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.433335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.435510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:15.450819 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.452700 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.471773 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1261/2511] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:15.481008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08344491-b5f3-402c-8c91-7131a25c8953 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:15.486177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:15.487703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:15.487703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:15.488607 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] PATH_INFO: `/v3/users/d47360e6c0854d6d93d90cafe670bd79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:15.490880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47360e6c0854d6d93d90cafe670bd79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:15.493281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:15.560587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47360e6c0854d6d93d90cafe670bd79'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.576878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47360e6c0854d6d93d90cafe670bd79'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.578010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47360e6c0854d6d93d90cafe670bd79'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.580641 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:15.588236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47360e6c0854d6d93d90cafe670bd79'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.590054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47360e6c0854d6d93d90cafe670bd79'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:15.605222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1251/2512] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:15 2022] DELETE /identity/v3/users/d47360e6c0854d6d93d90cafe670bd79 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:52:15.961851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:15.982214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc04c7-7289-4ba6-85fa-207544eec98c service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.058808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.059987 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.060405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.060790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] PATH_INFO: `/v3/projects/9bef378e0c1140a58c3727bd8d2a3e6e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.062053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bef378e0c1140a58c3727bd8d2a3e6e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.064518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.117120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bef378e0c1140a58c3727bd8d2a3e6e', 'request_id': 'req-8087052c-628a-4b7a-8cb5-7d0b52e3221f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.125744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bef378e0c1140a58c3727bd8d2a3e6e', 'request_id': 'req-8087052c-628a-4b7a-8cb5-7d0b52e3221f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.133779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bef378e0c1140a58c3727bd8d2a3e6e', 'request_id': 'req-8087052c-628a-4b7a-8cb5-7d0b52e3221f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.146212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] The token cache is being invalidate because project 9bef378e0c1140a58c3727bd8d2a3e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:16.146519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.147179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.148800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1262/2513] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:15 2022] DELETE /identity/v3/projects/9bef378e0c1140a58c3727bd8d2a3e6e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:16.153238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.154598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.168644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761404f-15f1-4171-9566-3c7f7c935d6f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.232616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.233528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.233877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.234159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] PATH_INFO: `/v3/users/99c28c8c1af4466ebc90fe361ec7264c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.235363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c28c8c1af4466ebc90fe361ec7264c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.237872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.287426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c28c8c1af4466ebc90fe361ec7264c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.303866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c28c8c1af4466ebc90fe361ec7264c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.304737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c28c8c1af4466ebc90fe361ec7264c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.313372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c28c8c1af4466ebc90fe361ec7264c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.315148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-04d0115f-2456-45ce-9b99-033f782d0972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c28c8c1af4466ebc90fe361ec7264c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.328730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1252/2514] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:16 2022] DELETE /identity/v3/users/99c28c8c1af4466ebc90fe361ec7264c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:16.608821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.610457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.624979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087052c-628a-4b7a-8cb5-7d0b52e3221f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.684657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.685517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.685800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.686076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] PATH_INFO: `/v3/projects/ae42a6f03c8b481a896f7137cb972fe0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.687245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae42a6f03c8b481a896f7137cb972fe0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.689396 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.712373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:16.740512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae42a6f03c8b481a896f7137cb972fe0', 'request_id': 'req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.747973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae42a6f03c8b481a896f7137cb972fe0', 'request_id': 'req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.755246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae42a6f03c8b481a896f7137cb972fe0', 'request_id': 'req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.765553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] The token cache is being invalidate because project ae42a6f03c8b481a896f7137cb972fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:16.765845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.766491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a2cff03c-b8b7-41f0-8ba1-e7fd0eac3ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:16.768568 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1263/2515] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:16 2022] DELETE /identity/v3/projects/ae42a6f03c8b481a896f7137cb972fe0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:16.774443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.774973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.775082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.793516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:16.794752 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.866173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1253/2516] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:16.875457 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.875748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.876030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.895014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:16.896229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:16.904178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1264/2517] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:16.908520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.913709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:16.921066 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a5d26-6c30-4ab4-92b7-3225edf77fe3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.927969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.928684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.929000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.929296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.930063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:52:16.930363 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.931954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.945058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1254/2518] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:52:16.949650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:16.959281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710129c-6c5e-4cf6-bd90-47c20ab76de8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:16.963825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:16.964504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:16.964786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:16.965059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:16.965749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:16.967643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:16.993839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1265/2519] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:16 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:16.997377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.008349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f4bfa-a093-47ac-9750-097fa88ed79c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.063635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.064338 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.064628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.064910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.066498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.073002 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.095601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1255/2520] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:16 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:17.099630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.110235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0a0e2-2532-4537-8468-1d047628d74f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.115571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.116241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.116559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.116860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.117591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.119242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.130602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1266/2521] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:17.134745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.145004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4a36c-70dd-4068-9929-d1a021331ab8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.149692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182e270-3d4b-4d12-b58b-01d82e2d8153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.150366 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e270-3d4b-4d12-b58b-01d82e2d8153 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.150651 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e270-3d4b-4d12-b58b-01d82e2d8153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.150932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e270-3d4b-4d12-b58b-01d82e2d8153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.151654 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e270-3d4b-4d12-b58b-01d82e2d8153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.153378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e270-3d4b-4d12-b58b-01d82e2d8153 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.164770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1256/2522] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:17.168954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.180212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81700645-207a-4dec-8d5e-226d25031c9e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.185119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.186002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.186283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.186557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] PATH_INFO: `/v3/projects/0ae942a993c7410fb164c6d7787b00ce/users/0c7d0cfd4ac84597990814a8d3e690c0/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.201513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae942a993c7410fb164c6d7787b00ce, user_id=0c7d0cfd4ac84597990814a8d3e690c0, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.204154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.217629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1267/2523] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:17 2022] PUT /identity/v3/projects/0ae942a993c7410fb164c6d7787b00ce/users/0c7d0cfd4ac84597990814a8d3e690c0/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:17.229919 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.230214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.230502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.259444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] MFA Rules not processed for user `0c7d0cfd4ac84597990814a8d3e690c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:17.260734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:17.324208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1257/2524] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:17.328740 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.392576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748c5e6-4144-4d8a-8039-5a9406f02530 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.397624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.398284 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.398630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.398916 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.399607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.401546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.428097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1268/2525] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:17 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:17.432507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.443726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcb8d5-911e-4da4-9e5b-4e173b2504e1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.495261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:17.502148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.503305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.503606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.503891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.504644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.506595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.531753 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1258/2526] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:17 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:52:17.536440 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.547034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1060a4-fa64-4495-b716-a3e73f134053 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.551923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.552669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.553001 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.553310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.554081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.555823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.566965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1269/2527] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:17.571572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.582510 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28017df-4301-44cc-8403-7fe10282045c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.590599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.591493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.591780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.592062 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] PATH_INFO: `/v3/projects/126ab84d00c24de785f4a42e53da5414/users/76e8044f01aa48ed83ef8f98910b0586/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.607667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=126ab84d00c24de785f4a42e53da5414, user_id=76e8044f01aa48ed83ef8f98910b0586, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.610367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.623128 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1259/2528] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:17 2022] PUT /identity/v3/projects/126ab84d00c24de785f4a42e53da5414/users/76e8044f01aa48ed83ef8f98910b0586/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:17.627092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.637166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc8f5e-d92f-4663-b2ab-b650b54723bb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.650966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:17.699813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f738b9-82e8-405c-a2ab-211d10eaccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.700532 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f738b9-82e8-405c-a2ab-211d10eaccb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.700821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f738b9-82e8-405c-a2ab-211d10eaccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.701099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f738b9-82e8-405c-a2ab-211d10eaccb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.701817 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f738b9-82e8-405c-a2ab-211d10eaccb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.703437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f738b9-82e8-405c-a2ab-211d10eaccb6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.718432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1270/2529] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:17.723344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.734625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570df8e-92af-49bd-8aa3-eae4f4298d2a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.739454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.740376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.740666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.740947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] PATH_INFO: `/v3/projects/126ab84d00c24de785f4a42e53da5414/users/76e8044f01aa48ed83ef8f98910b0586/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.743060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126ab84d00c24de785f4a42e53da5414, user_id=76e8044f01aa48ed83ef8f98910b0586, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.745844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.757206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1260/2530] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:17 2022] PUT /identity/v3/projects/126ab84d00c24de785f4a42e53da5414/users/76e8044f01aa48ed83ef8f98910b0586/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:17.767549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.767915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.768795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.796615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] MFA Rules not processed for user `76e8044f01aa48ed83ef8f98910b0586`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:17.797794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:17.870244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1271/2531] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:17.882192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:17.883908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:17.942000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e3ea-36fc-49d0-baac-14316d6d8181 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:17.945954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:17.946590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:17.946869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:17.947143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:17.956966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:17.958475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:17.967674 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1261/2532] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:18.051315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:18.061068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f39b0-7ae8-46f8-9aa6-0ce70b78f469 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:18.064749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95092ea4-92a2-4b2e-8392-7f6ded1e469a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:18.065949 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95092ea4-92a2-4b2e-8392-7f6ded1e469a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:18.066239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95092ea4-92a2-4b2e-8392-7f6ded1e469a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:18.066520 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95092ea4-92a2-4b2e-8392-7f6ded1e469a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:18.123066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95092ea4-92a2-4b2e-8392-7f6ded1e469a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:18.124616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95092ea4-92a2-4b2e-8392-7f6ded1e469a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:18.134000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1272/2533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:22.822313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:22.885107 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7963a4-b9da-49a3-851c-24657d0397e6 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:22.889633 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:22.890502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:22.890787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:22.891067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] PATH_INFO: `/v3/users/0c7d0cfd4ac84597990814a8d3e690c0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:22.892252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7d0cfd4ac84597990814a8d3e690c0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:22.894335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:22.939656 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7d0cfd4ac84597990814a8d3e690c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:22.945811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:22.954625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7d0cfd4ac84597990814a8d3e690c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:22.955501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7d0cfd4ac84597990814a8d3e690c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:22.962713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7d0cfd4ac84597990814a8d3e690c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:22.963353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7d0cfd4ac84597990814a8d3e690c0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:22.975921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1262/2534] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:22 2022] DELETE /identity/v3/users/0c7d0cfd4ac84597990814a8d3e690c0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:22.980638 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:22.981097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:22.981208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:22.999194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:23.000442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:23.033693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:23.061919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1273/2535] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:23.072165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:23.073612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:23.131912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b2abf-7919-455e-96d0-908dc3b73ff0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:23.135565 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:23.136226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:23.136548 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:23.136826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:23.146443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:23.147881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:23.157594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1263/2536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:23.433714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:23.435250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:23.448027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ce30d-e4d2-4a6c-9488-9e429e55eb94 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:23.452689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:23.453547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:23.453829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:23.454104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] PATH_INFO: `/v3/projects/0ae942a993c7410fb164c6d7787b00ce` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:23.455384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae942a993c7410fb164c6d7787b00ce)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:23.457910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:23.507807 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae942a993c7410fb164c6d7787b00ce', 'request_id': 'req-1a517331-a068-413b-a8a1-3d8a929ad906'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.515089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae942a993c7410fb164c6d7787b00ce', 'request_id': 'req-1a517331-a068-413b-a8a1-3d8a929ad906'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.521682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae942a993c7410fb164c6d7787b00ce', 'request_id': 'req-1a517331-a068-413b-a8a1-3d8a929ad906'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.531918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] The token cache is being invalidate because project 0ae942a993c7410fb164c6d7787b00ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:23.532204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.532861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.534469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1274/2537] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:23 2022] DELETE /identity/v3/projects/0ae942a993c7410fb164c6d7787b00ce => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:52:23.538612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:23.548264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcf61c-2261-4f8c-939c-8459a6ef749a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:23.613122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:23.613990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:23.614303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:23.614577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] PATH_INFO: `/v3/users/76e8044f01aa48ed83ef8f98910b0586` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:23.615754 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e8044f01aa48ed83ef8f98910b0586)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:23.617715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:23.665301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e8044f01aa48ed83ef8f98910b0586'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.677992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e8044f01aa48ed83ef8f98910b0586'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.678636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e8044f01aa48ed83ef8f98910b0586'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.685796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e8044f01aa48ed83ef8f98910b0586'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.686421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f978dff3-0824-4b7d-9498-56ca931ab4bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e8044f01aa48ed83ef8f98910b0586'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:23.691778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:23.702215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1264/2538] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:23 2022] DELETE /identity/v3/users/76e8044f01aa48ed83ef8f98910b0586 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:23.987675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:23.989158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:24.004579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a517331-a068-413b-a8a1-3d8a929ad906 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.062096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.062994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.063315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.063601 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] PATH_INFO: `/v3/projects/126ab84d00c24de785f4a42e53da5414` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.064877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=126ab84d00c24de785f4a42e53da5414)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.066889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.114009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126ab84d00c24de785f4a42e53da5414', 'request_id': 'req-971e8189-da5b-4970-9adb-29c8de3bfd5c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:24.121668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126ab84d00c24de785f4a42e53da5414', 'request_id': 'req-971e8189-da5b-4970-9adb-29c8de3bfd5c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:24.128255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126ab84d00c24de785f4a42e53da5414', 'request_id': 'req-971e8189-da5b-4970-9adb-29c8de3bfd5c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:24.130270 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:24.140702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] The token cache is being invalidate because project 126ab84d00c24de785f4a42e53da5414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:24.141000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:24.141655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-971e8189-da5b-4970-9adb-29c8de3bfd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:24.143364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1275/2539] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:23 2022] DELETE /identity/v3/projects/126ab84d00c24de785f4a42e53da5414 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:24.148218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.148719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.148816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.168899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:24.170111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:24.245891 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1265/2540] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:24.255514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.255804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.256199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.276097 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:24.277354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:24.285931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1276/2541] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:52:24.290207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.299914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196fb0-3efe-4426-920e-a2a23a7e8774 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.304559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.305215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.305499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.305774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.306499 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:52:24.306805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.308445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.320533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1266/2542] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:52:24.324788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.335229 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7523e-d2aa-41ec-adec-a2669b585f02 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.340222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.340925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.341212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.341511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.342226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.344144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.372375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1277/2543] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:24 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:24.377356 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.387612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f1cd-15f7-4e47-bd7a-efe675c98ad3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.444555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.445227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.445521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.445803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.446488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.448486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.470028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1267/2544] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:24 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:24.474744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.485065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd208b0-7a6a-4eef-896e-9fad451c322b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.489765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.490421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.490704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.490983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.491701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.493349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.503861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1278/2545] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:24.507786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.518278 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae564c-91dc-465d-bdee-364d158007ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.522960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc16f2b-8570-46d7-80f7-680e85afeb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.523631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc16f2b-8570-46d7-80f7-680e85afeb5f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.523909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc16f2b-8570-46d7-80f7-680e85afeb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.524193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc16f2b-8570-46d7-80f7-680e85afeb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.524951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc16f2b-8570-46d7-80f7-680e85afeb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.526573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc16f2b-8570-46d7-80f7-680e85afeb5f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.537437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1268/2546] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:24.541533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.552067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bbb44-a50e-43bb-96c5-717e4264767b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.559223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.560778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.561223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.561675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] PATH_INFO: `/v3/projects/3a26e502aebb47dfac9328330f4bd6d7/users/fa96256e67a547f49e7a46107f388ab1/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.582570 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a26e502aebb47dfac9328330f4bd6d7, user_id=fa96256e67a547f49e7a46107f388ab1, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.585844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.602756 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1279/2547] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:24 2022] PUT /identity/v3/projects/3a26e502aebb47dfac9328330f4bd6d7/users/fa96256e67a547f49e7a46107f388ab1/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:24.613668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.613966 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.614252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.645598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] MFA Rules not processed for user `fa96256e67a547f49e7a46107f388ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:24.646842 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:24.694342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:24.713904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1269/2548] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:24.724854 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:24.726521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:24.784027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5202-0815-4026-8dbd-cfccab662b47 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:24.787614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162e727-21ef-4af9-913e-f41ff17e0151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:24.788259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e727-21ef-4af9-913e-f41ff17e0151 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:24.788576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e727-21ef-4af9-913e-f41ff17e0151 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:24.788852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e727-21ef-4af9-913e-f41ff17e0151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:24.797175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162e727-21ef-4af9-913e-f41ff17e0151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:24.798605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162e727-21ef-4af9-913e-f41ff17e0151 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:24.807432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1280/2549] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:27.852264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:27.915146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b4ca-9a64-4626-acf7-72e65e5893ae None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:27.919801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:27.920759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:27.921050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:27.921333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] PATH_INFO: `/v3/users/fa96256e67a547f49e7a46107f388ab1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:27.922660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa96256e67a547f49e7a46107f388ab1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:27.925291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:27.977696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa96256e67a547f49e7a46107f388ab1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:27.990594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa96256e67a547f49e7a46107f388ab1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:27.991273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa96256e67a547f49e7a46107f388ab1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:27.999465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa96256e67a547f49e7a46107f388ab1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.000091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa96256e67a547f49e7a46107f388ab1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.012973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1270/2550] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:52:27 2022] DELETE /identity/v3/users/fa96256e67a547f49e7a46107f388ab1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:28.017522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.018504 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.018810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.036792 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:28.038027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:28.098179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1281/2551] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:28.107804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.109247 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:28.168074 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055a1f-8e8d-410e-b396-d79ad45257f6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.171700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf2305-c3be-478b-a2d5-2f268b06b3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.172373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf2305-c3be-478b-a2d5-2f268b06b3cd service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.172659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf2305-c3be-478b-a2d5-2f268b06b3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.172935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf2305-c3be-478b-a2d5-2f268b06b3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.182046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf2305-c3be-478b-a2d5-2f268b06b3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.183482 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf2305-c3be-478b-a2d5-2f268b06b3cd service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.193041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1271/2552] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:52:28.468387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.469911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:28.482957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649572d1-6372-4c1e-bc8d-4ac2ed26f8f8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.488739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.489623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.489907 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.490186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] PATH_INFO: `/v3/projects/3a26e502aebb47dfac9328330f4bd6d7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.491423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a26e502aebb47dfac9328330f4bd6d7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.493392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.525948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:28.543682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a26e502aebb47dfac9328330f4bd6d7', 'request_id': 'req-12fc13ec-216f-45d5-aded-c0b77e0d128c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.551657 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a26e502aebb47dfac9328330f4bd6d7', 'request_id': 'req-12fc13ec-216f-45d5-aded-c0b77e0d128c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.558315 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a26e502aebb47dfac9328330f4bd6d7', 'request_id': 'req-12fc13ec-216f-45d5-aded-c0b77e0d128c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.569513 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] The token cache is being invalidate because project 3a26e502aebb47dfac9328330f4bd6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:52:28.569810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.570443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-12fc13ec-216f-45d5-aded-c0b77e0d128c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:52:28.571972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1282/2553] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:52:28 2022] DELETE /identity/v3/projects/3a26e502aebb47dfac9328330f4bd6d7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:28.576894 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.577188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.577468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.595728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:28.596973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:28.602266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:28.658668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1272/2554] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:28.667895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.668185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.668509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.686998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:28.688190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:28.696867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1283/2555] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:28.701206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.716017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b60de-a40c-432a-bb3c-d50ab51fae9c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.721230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.721905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.722194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.722477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.723208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:52:28.723521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.725191 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.737611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1273/2556] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:52:28.742003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.751266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce78543-c99a-4da3-9c76-49eaf23b30eb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.757155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.757816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.758103 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.758546 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.759255 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.761267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.788146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1284/2557] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:52:28 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:28.792100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.802631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213a71c-1e23-4d80-acec-0ab5e780ad54 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.858437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.859126 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.859405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.859690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.860415 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.862974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.883224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1274/2558] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:52:28 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:52:28.887351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.897894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4487c99-22b5-433f-93c1-57cdad49dd38 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.902474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.903128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.903421 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.903694 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.904463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.906054 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.920969 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1285/2559] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:28.923627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.934287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa443b1-0dc9-4ef2-bfe9-cbe5e39908ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.938911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958aef9-6070-4573-b2ae-2bbbbe2dd349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.940383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958aef9-6070-4573-b2ae-2bbbbe2dd349 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.940720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958aef9-6070-4573-b2ae-2bbbbe2dd349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.941032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958aef9-6070-4573-b2ae-2bbbbe2dd349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.941755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958aef9-6070-4573-b2ae-2bbbbe2dd349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.943626 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958aef9-6070-4573-b2ae-2bbbbe2dd349 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:28.953579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1275/2560] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:52:28.957710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:28.967562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f7a5c-c0a7-47f4-ac79-471feb14e0e8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:28.972254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:28.973156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:28.973441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:28.973728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] PATH_INFO: `/v3/projects/bc896b735afc4652830a1803f9a3f52b/users/0b57ecc0c80949ef83f89443bceb1897/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:28.988626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc896b735afc4652830a1803f9a3f52b, user_id=0b57ecc0c80949ef83f89443bceb1897, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:28.991204 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:29.005923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1286/2561] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:52:28 2022] PUT /identity/v3/projects/bc896b735afc4652830a1803f9a3f52b/users/0b57ecc0c80949ef83f89443bceb1897/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:52:29.016184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:29.016508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:29.016796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:29.043246 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] MFA Rules not processed for user `0b57ecc0c80949ef83f89443bceb1897`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:52:29.044494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:29.107337 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1276/2562] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:52:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:52:29.119365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:29.121303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:29.178945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba348ab-c728-490f-905f-83a4b64390bb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:29.182544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:29.183196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:29.183479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:29.183757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:29.192082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:29.193838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:29.202718 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1287/2563] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:39.104584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:39.198641 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:52:47.748883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:52:47.750400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:47.821610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00299-daba-4911-9c98-bce299d4f411 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:52:47.826518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:52:47.827163 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:52:47.827443 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:52:47.827719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:52:47.829488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:52:47.841560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:52:47.843130 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade401dc-8d3e-45ee-820b-bab95ca8868c service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:52:47.854166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1277/2564] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:52:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:52:57.849684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:04.239015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:04.303099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e59b6-7571-407b-a385-1963c150981c service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:04.307671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:04.308569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:04.308857 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:04.309139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] PATH_INFO: `/v3/users/0b57ecc0c80949ef83f89443bceb1897` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:04.310323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b57ecc0c80949ef83f89443bceb1897)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:04.312243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:04.359851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b57ecc0c80949ef83f89443bceb1897'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.360554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b57ecc0c80949ef83f89443bceb1897'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.367646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b57ecc0c80949ef83f89443bceb1897'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.368420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b57ecc0c80949ef83f89443bceb1897'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.376622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b57ecc0c80949ef83f89443bceb1897'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.395206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1288/2565] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:53:04 2022] DELETE /identity/v3/users/0b57ecc0c80949ef83f89443bceb1897 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:04.399723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:04.400193 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:04.400290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:04.419390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:04.420691 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:04.480332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1278/2566] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:04.490955 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:04.492438 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:04.550215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805848e8-a807-4539-b8c1-f66f68d495ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:04.554140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3c67b-3063-4e53-9b10-2d264de8c36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:04.554806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3c67b-3063-4e53-9b10-2d264de8c36e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:04.555094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3c67b-3063-4e53-9b10-2d264de8c36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:04.555379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3c67b-3063-4e53-9b10-2d264de8c36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:04.564822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3c67b-3063-4e53-9b10-2d264de8c36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:04.566313 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3c67b-3063-4e53-9b10-2d264de8c36e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:04.576074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1289/2567] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:04.850256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:04.851918 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:04.870168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf8056-800b-4636-a1d1-31afe0b4c52d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:04.876146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:04.877239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:04.877552 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:04.877958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] PATH_INFO: `/v3/projects/bc896b735afc4652830a1803f9a3f52b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:04.879357 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc896b735afc4652830a1803f9a3f52b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:04.881328 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:04.934799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc896b735afc4652830a1803f9a3f52b', 'request_id': 'req-448babd7-1172-43ba-a8ff-5af9b9d7b530'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.940152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc896b735afc4652830a1803f9a3f52b', 'request_id': 'req-448babd7-1172-43ba-a8ff-5af9b9d7b530'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.948180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc896b735afc4652830a1803f9a3f52b', 'request_id': 'req-448babd7-1172-43ba-a8ff-5af9b9d7b530'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.960887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] The token cache is being invalidate because project bc896b735afc4652830a1803f9a3f52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:53:04.961322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.962608 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-448babd7-1172-43ba-a8ff-5af9b9d7b530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:04.963872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1279/2568] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:53:04 2022] DELETE /identity/v3/projects/bc896b735afc4652830a1803f9a3f52b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:04.968779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:04.969071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:04.969357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:04.987783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:04.989071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:05.007473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:05.052900 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1290/2569] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:05.060720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.061014 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.061321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.079782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:05.081775 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:05.089883 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1280/2570] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:05.094162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.104860 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98159-fbac-4c00-a0d7-3aa3d7c8b89d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.109728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.110778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.110778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.111412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.111659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:53:05.111972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.113867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.126728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1291/2571] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:53:05.132368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.142309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d7d72-2a8f-4e05-b4c6-3559ef119cf9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.148012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.148832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.149204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.149497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.150258 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.152418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.178271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1281/2572] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:53:05 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:53:05.182473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.193725 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0d7b6-9641-4356-86b9-6c6eaa2dc5fd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.251309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.253161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.253452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.253738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.254505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.256409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.278246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1292/2573] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:53:05 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:53:05.282543 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.294521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f02936-74f2-4dee-abba-44c4748203f1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.300170 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b0096-3309-45f5-a126-54976928139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.300900 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b0096-3309-45f5-a126-54976928139e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.301187 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b0096-3309-45f5-a126-54976928139e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.301463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b0096-3309-45f5-a126-54976928139e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.302203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b0096-3309-45f5-a126-54976928139e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.303896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b0096-3309-45f5-a126-54976928139e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.317173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1282/2574] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:53:05.321409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.333011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc72-28b5-40e6-8ee2-46eaed31c681 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.338129 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822df21-1a69-4e9e-8343-7e544c170eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.338813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822df21-1a69-4e9e-8343-7e544c170eca admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.339114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822df21-1a69-4e9e-8343-7e544c170eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.339389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822df21-1a69-4e9e-8343-7e544c170eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.340122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822df21-1a69-4e9e-8343-7e544c170eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.341771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822df21-1a69-4e9e-8343-7e544c170eca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.355616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1293/2575] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:53:05.357026 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b0096-3309-45f5-a126-54976928139e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.367414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b0096-3309-45f5-a126-54976928139e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.379068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.380012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.380296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.380604 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] PATH_INFO: `/v3/projects/59670eb6faa3412ebe4cee53593aa3ae/users/b9c040b6259e4ec79ac66e670edcb6d8/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.398886 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59670eb6faa3412ebe4cee53593aa3ae, user_id=b9c040b6259e4ec79ac66e670edcb6d8, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.401794 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.415549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1283/2576] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:53:05 2022] PUT /identity/v3/projects/59670eb6faa3412ebe4cee53593aa3ae/users/b9c040b6259e4ec79ac66e670edcb6d8/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:05.427663 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.427957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.428279 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.457996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] MFA Rules not processed for user `b9c040b6259e4ec79ac66e670edcb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:05.459217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:05.524179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1294/2577] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:05.535049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:05.536795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:05.558662 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:05.596216 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5fa8-2f29-487e-81b7-82ad3775c651 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:05.599912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c4cc2-8ebb-44bf-bc4f-9a44340644aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:05.600579 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c4cc2-8ebb-44bf-bc4f-9a44340644aa service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:05.600862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c4cc2-8ebb-44bf-bc4f-9a44340644aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:05.601134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c4cc2-8ebb-44bf-bc4f-9a44340644aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:05.609502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c4cc2-8ebb-44bf-bc4f-9a44340644aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:05.611210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c4cc2-8ebb-44bf-bc4f-9a44340644aa service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:05.620414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1284/2578] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:53:10.602744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:10.604164 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:10.624721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:10.665834 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359f5f-23cc-4b2c-b562-22b47ab3745f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:10.669630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:10.670293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:10.670582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:10.670865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:10.672736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:10.750035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:10.752882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:10.769359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1295/2579] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:10.996485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:10.997007 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:10.997359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:11.021248 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:11.025283 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:11.128046 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1285/2580] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:53:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:11.208393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:11.210407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:11.302226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bf7db-a9da-4ef1-9bf8-1a44521cd02a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:11.307749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:11.308748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:11.309138 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:11.309671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:11.320918 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:11.322730 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:11.334633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1296/2581] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:53:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:53:16.561578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:16.563620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:16.586487 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe657-568e-4b9c-9fb9-64b6ff2aee9a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:16.592491 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:16.593634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:16.593956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:16.594399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:16.599029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:16.684864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:16.686381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e47ad3-f129-4918-ad55-8be9e561b799 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:16.699644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1286/2582] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:17.482533 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:17.498442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c72e06-dc89-480a-a6dc-062a09407a66 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:17.511285 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:17.513428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:17.513719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:17.514009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] PATH_INFO: `/v3/users/b9c040b6259e4ec79ac66e670edcb6d8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:17.515411 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c040b6259e4ec79ac66e670edcb6d8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:17.517946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:17.587924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c040b6259e4ec79ac66e670edcb6d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:17.588906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c040b6259e4ec79ac66e670edcb6d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:17.598762 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c040b6259e4ec79ac66e670edcb6d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:17.599770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c040b6259e4ec79ac66e670edcb6d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:17.601789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:17.610509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c040b6259e4ec79ac66e670edcb6d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:17.645130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1297/2583] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:53:17 2022] DELETE /identity/v3/users/b9c040b6259e4ec79ac66e670edcb6d8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:17.646657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:17.647162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:17.647245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:17.679840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:17.681939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:17.784222 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1287/2584] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:17.800735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:17.807676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:17.929808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3838d9-14ef-4ce7-aeb2-aa7f8389909e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:17.937126 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf095ec-9079-49a6-8054-c58565f6f90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:17.938293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf095ec-9079-49a6-8054-c58565f6f90e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:17.939385 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf095ec-9079-49a6-8054-c58565f6f90e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:17.939967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf095ec-9079-49a6-8054-c58565f6f90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:17.962137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf095ec-9079-49a6-8054-c58565f6f90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:17.965902 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf095ec-9079-49a6-8054-c58565f6f90e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:18.004434 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1298/2585] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:18.574256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:18.578178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:18.586750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:18.596600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feeb93-ae7e-4a93-9e03-f567be289d13 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:18.627464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:18.629807 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:18.630708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:18.630708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] PATH_INFO: `/v3/projects/59670eb6faa3412ebe4cee53593aa3ae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:18.634789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] RBAC: Authorizing `identity:delete_project(project_id=59670eb6faa3412ebe4cee53593aa3ae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:18.636869 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:18.696909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59670eb6faa3412ebe4cee53593aa3ae', 'request_id': 'req-d4dd32e2-7c11-4917-a4bd-f1c93215908c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:18.707297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59670eb6faa3412ebe4cee53593aa3ae', 'request_id': 'req-d4dd32e2-7c11-4917-a4bd-f1c93215908c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:18.734344 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59670eb6faa3412ebe4cee53593aa3ae', 'request_id': 'req-d4dd32e2-7c11-4917-a4bd-f1c93215908c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:18.755211 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] The token cache is being invalidate because project 59670eb6faa3412ebe4cee53593aa3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:53:18.755810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:18.757043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d4dd32e2-7c11-4917-a4bd-f1c93215908c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:18.761864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1288/2586] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:53:18 2022] DELETE /identity/v3/projects/59670eb6faa3412ebe4cee53593aa3ae => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:18.767897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:18.768465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:18.768782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:18.797079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:18.798349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:18.892512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1299/2587] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:18.899252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:18.899590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:18.899821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:18.928840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:18.929591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:18.953220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1289/2588] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:18.958106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:18.968020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98e4e0-937b-4a86-96df-b6dd40e8212c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:18.986394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:18.987628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:18.988468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:18.988468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:18.989988 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:53:18.990303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:18.992182 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.013596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:19.019766 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1300/2589] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:53:19.024019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6f034-e018-43ae-b67d-ffd53b722c48 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:19.037908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:19.038798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.039086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.039371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.040077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:19.042077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.120151 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1290/2590] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:53:19 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:53:19.123507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:19.142057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbf1c-1c50-41ea-a962-c648b5985017 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:19.246452 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:19.249585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.252983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.256184 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.258233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:19.261017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.273723 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:19.312106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1301/2591] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:53:19 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:53:19.313400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:19.331429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611ec47-9fec-4838-a6d1-dbee0a24223c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:19.343786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:19.344771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.345064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.345444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.346514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:19.348364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.383786 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1291/2592] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:53:19.386736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:19.410706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5273-e2dc-4dc1-9a64-3b33cb6a5354 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:19.426146 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5c0a0-d97d-49f3-b08f-84a344f788dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:19.426942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5c0a0-d97d-49f3-b08f-84a344f788dd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.427420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5c0a0-d97d-49f3-b08f-84a344f788dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.427753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5c0a0-d97d-49f3-b08f-84a344f788dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.429544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5c0a0-d97d-49f3-b08f-84a344f788dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:19.434235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5c0a0-d97d-49f3-b08f-84a344f788dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.458857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:19.459491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1302/2593] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:53:19.472679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62846e5-178c-4983-921e-9ec4dd4f24d8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:19.478553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:19.479851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.479851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.480058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] PATH_INFO: `/v3/projects/189885bf9a5944bf8eaedd89bb5bec9f/users/f98684788906448bb3b35af5a2fc017f/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.499776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189885bf9a5944bf8eaedd89bb5bec9f, user_id=f98684788906448bb3b35af5a2fc017f, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:19.504413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.525680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1292/2594] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:53:19 2022] PUT /identity/v3/projects/189885bf9a5944bf8eaedd89bb5bec9f/users/f98684788906448bb3b35af5a2fc017f/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:53:19.530446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.530736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.531016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.553658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:19.555950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:19.637366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1303/2595] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:19.648605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:19.650434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:19.741417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ef864-42d9-4073-9ce2-0df7f39fb715 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:19.747117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb332c-d794-484f-8045-cd6450f4c3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:19.748232 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb332c-d794-484f-8045-cd6450f4c3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:19.748780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb332c-d794-484f-8045-cd6450f4c3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:19.749309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb332c-d794-484f-8045-cd6450f4c3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:19.761404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb332c-d794-484f-8045-cd6450f4c3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:19.763629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb332c-d794-484f-8045-cd6450f4c3a1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:19.801460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1293/2596] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:21.426664 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:21.429499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:21.541535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6264b7-6d5a-4ce1-a03e-541a5a9b0a58 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:21.547332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:21.548476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:21.549443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:21.549443 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:21.552374 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:21.572042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:21.574830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:21.593549 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1304/2597] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:26.959259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:26.959830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:26.959927 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.005217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] MFA Rules not processed for user `f98684788906448bb3b35af5a2fc017f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:27.006933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:27.037184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:27.119050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1294/2598] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:27.120381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.136721 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e9147-2bb3-4705-891b-9dc59e1c7fec service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.141867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.142650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.143008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.144226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.145092 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.147636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.189419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1305/2599] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:53:27 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:53:27.193502 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.207988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f36d28-bc31-4562-9640-23cd15ede310 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.294853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.295708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.296098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.296511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.297325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.299702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.340534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1295/2600] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:53:27 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:53:27.346951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.361467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02594ab0-f17c-4f99-898f-30272d9a5a37 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.368911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.369763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.370155 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.370537 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.371525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.373593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.388511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1306/2601] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:53:27.392687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.406476 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1619e5b-e97e-4cc4-b611-826e218a1614 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.412243 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.413422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.413804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.414183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] PATH_INFO: `/v3/projects/19c7ce863a774fd6bd3d3cd5d8767ac9/users/d6cbdab41f8644618d187cac794c8244/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.432197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c7ce863a774fd6bd3d3cd5d8767ac9, user_id=d6cbdab41f8644618d187cac794c8244, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.435125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.455261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1296/2602] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:53:27 2022] PUT /identity/v3/projects/19c7ce863a774fd6bd3d3cd5d8767ac9/users/d6cbdab41f8644618d187cac794c8244/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:53:27.459946 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.472190 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c4b9c-47c2-495e-90f4-caef2a8baa11 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.535696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:27.558312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.559119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.559478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.559876 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.560702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.562394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.577144 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1307/2603] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:53:27.578406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.589715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc0fe-0c59-4efb-b371-dbabf88688d4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.594973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.595873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.596162 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.596787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] PATH_INFO: `/v3/projects/19c7ce863a774fd6bd3d3cd5d8767ac9/users/d6cbdab41f8644618d187cac794c8244/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.599041 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c7ce863a774fd6bd3d3cd5d8767ac9, user_id=d6cbdab41f8644618d187cac794c8244, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.601780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.612837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:27.614558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:27.615039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1297/2604] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:53:27 2022] PUT /identity/v3/projects/19c7ce863a774fd6bd3d3cd5d8767ac9/users/d6cbdab41f8644618d187cac794c8244/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:27.628348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca075e-906f-400f-a228-fbc3d3f14b3d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:27.707530 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:27.708769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:27.708769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:27.709068 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] PATH_INFO: `/v3/users/bdb847efec494be8a5a89da456f54d3d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:27.710489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb847efec494be8a5a89da456f54d3d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:27.712711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:27.787172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb847efec494be8a5a89da456f54d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:27.788186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb847efec494be8a5a89da456f54d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:27.796356 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb847efec494be8a5a89da456f54d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:27.797148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb847efec494be8a5a89da456f54d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:27.806239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb847efec494be8a5a89da456f54d3d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:27.829542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1308/2605] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:53:27 2022] DELETE /identity/v3/users/bdb847efec494be8a5a89da456f54d3d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:53:28.052824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:28.054433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:28.073451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd449d2-d3cc-4028-b811-a88280011fc2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:28.162687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:28.163595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:28.163885 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:28.164166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] PATH_INFO: `/v3/projects/c85800d659ab43ff9b16266f9f037fc2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:28.165485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85800d659ab43ff9b16266f9f037fc2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:28.167496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:28.177501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:28.230747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85800d659ab43ff9b16266f9f037fc2', 'request_id': 'req-3b52b945-3e03-4b06-9d36-147a2f367bfb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.236990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85800d659ab43ff9b16266f9f037fc2', 'request_id': 'req-3b52b945-3e03-4b06-9d36-147a2f367bfb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.245635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85800d659ab43ff9b16266f9f037fc2', 'request_id': 'req-3b52b945-3e03-4b06-9d36-147a2f367bfb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.257986 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] The token cache is being invalidate because project c85800d659ab43ff9b16266f9f037fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:53:28.258285 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.258941 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.260584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1298/2606] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:53:28 2022] DELETE /identity/v3/projects/c85800d659ab43ff9b16266f9f037fc2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:28.264658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:28.274880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ad76b-7438-4c9e-87a6-bc27f85d522b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:28.352166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:28.361683 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:28.362870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:28.363479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:28.363960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] PATH_INFO: `/v3/users/d165790929f04ad79560baff5225704b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:28.371501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d165790929f04ad79560baff5225704b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:28.373815 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:28.442751 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd165790929f04ad79560baff5225704b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.443500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd165790929f04ad79560baff5225704b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.450993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd165790929f04ad79560baff5225704b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.451686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd165790929f04ad79560baff5225704b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.460508 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d23d4df7-8b68-44fb-958d-97efdc1652d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd165790929f04ad79560baff5225704b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.484711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1309/2607] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:53:28 2022] DELETE /identity/v3/users/d165790929f04ad79560baff5225704b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:28.773101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:28.775102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:28.804192 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b945-3e03-4b06-9d36-147a2f367bfb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:28.874983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:28.876435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:28.877148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:28.878586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] PATH_INFO: `/v3/projects/ca93026e5b7e450e851a230162a45408` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:28.880332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca93026e5b7e450e851a230162a45408)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:28.882452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:28.952480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca93026e5b7e450e851a230162a45408', 'request_id': 'req-166fbb3b-fe3d-41d3-95b7-b981c8299add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.968526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca93026e5b7e450e851a230162a45408', 'request_id': 'req-166fbb3b-fe3d-41d3-95b7-b981c8299add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.976870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca93026e5b7e450e851a230162a45408', 'request_id': 'req-166fbb3b-fe3d-41d3-95b7-b981c8299add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.995422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] The token cache is being invalidate because project ca93026e5b7e450e851a230162a45408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:53:28.995791 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:28.996409 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:53:29.013208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1299/2608] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:53:28 2022] DELETE /identity/v3/projects/ca93026e5b7e450e851a230162a45408 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:53:33.577218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4cff-50ef-4eab-a32c-bfee29fe069e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:33.577753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4cff-50ef-4eab-a32c-bfee29fe069e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:33.577753 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4cff-50ef-4eab-a32c-bfee29fe069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:33.613501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c36d4cff-50ef-4eab-a32c-bfee29fe069e None None] MFA Rules not processed for user `d6cbdab41f8644618d187cac794c8244`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:33.615489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c36d4cff-50ef-4eab-a32c-bfee29fe069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:33.730959 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1310/2609] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:33.745089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:33.752599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:33.812195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:33.873350 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fbb3b-fe3d-41d3-95b7-b981c8299add admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:33.880390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:33.881003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:33.881281 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:33.881574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:33.883374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:33.959879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062280-e36b-4ab6-8d92-eb02cbcfdedb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:33.960298 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062280-e36b-4ab6-8d92-eb02cbcfdedb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:33.960932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062280-e36b-4ab6-8d92-eb02cbcfdedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:33.970512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:33.973553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:33.984229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1300/2610] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:53:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:53:33.991179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-df062280-e36b-4ab6-8d92-eb02cbcfdedb None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:33.992469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df062280-e36b-4ab6-8d92-eb02cbcfdedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:34.088178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1311/2611] 10.222.0.16 () {58 vars in 1009 bytes} [Thu May 26 13:53:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:34.102655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:34.104119 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:34.211271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5e85f-31b2-4b11-a3b3-46ef28e9ee23 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:34.217095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ce995-04b2-4957-8383-cea87974146b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:34.217616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ce995-04b2-4957-8383-cea87974146b service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:34.217902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ce995-04b2-4957-8383-cea87974146b service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:34.218204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ce995-04b2-4957-8383-cea87974146b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:34.229748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ce995-04b2-4957-8383-cea87974146b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:34.231381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ce995-04b2-4957-8383-cea87974146b service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:34.240820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1301/2612] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:53:44.063075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:44.236862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=76 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:53:56.634562 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7a0c-7273-4705-a9c6-7138f4f61309 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:56.635042 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7a0c-7273-4705-a9c6-7138f4f61309 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:56.635150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7a0c-7273-4705-a9c6-7138f4f61309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:56.653512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-06af7a0c-7273-4705-a9c6-7138f4f61309 None None] MFA Rules not processed for user `53d524fa52bd42cea09081345ff8d5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:56.654728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06af7a0c-7273-4705-a9c6-7138f4f61309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:56.662486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1312/2613] 10.222.0.16 () {58 vars in 967 bytes} [Thu May 26 13:53:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:53:56.674632 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ce995-04b2-4957-8383-cea87974146b service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:53:56.685659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ce995-04b2-4957-8383-cea87974146b service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:53:56.690903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4e2c8a980714257b8df1d486d446eeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:53:56.691549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:56.691834 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:56.692112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:56.700621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:53:56.702252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:53:56.710545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1302/2614] 10.222.0.16 () {60 vars in 1384 bytes} [Thu May 26 13:53:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 13:53:58.479463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:53:58.479998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:53:58.479998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:53:58.498340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] MFA Rules not processed for user `f98684788906448bb3b35af5a2fc017f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:53:58.499660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:53:58.507620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1313/2615] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:06.708101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:08.492935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:16.152830 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:16.154268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:16.226692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdccc9-5085-4034-aafa-6d5dff8fb4aa service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:16.230848 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:16.231880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:16.231880 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:16.232103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:16.244420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:16.245928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:16.258318 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1303/2616] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:21.219003 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:21.228882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc222dd-beed-4d20-bab6-e75d477d343b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:21.232404 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:21.233053 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:21.233339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:21.233616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:21.235389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:21.299273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:21.301523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:21.311239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1314/2617] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:24.446666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:24.448347 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:24.466331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee21ac-5f1b-428c-8b71-a8d55ce69474 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:24.471050 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:24.471939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:24.472937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:24.473369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] PATH_INFO: `/v3/users/9958aaa8690c468da2acec47b2bb5f23` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:24.474720 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9958aaa8690c468da2acec47b2bb5f23)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:24.476729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:24.529672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9958aaa8690c468da2acec47b2bb5f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.541887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9958aaa8690c468da2acec47b2bb5f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.542605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9958aaa8690c468da2acec47b2bb5f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.552032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9958aaa8690c468da2acec47b2bb5f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.552729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92ac5ced-4123-4b71-aee6-daf6048c29a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9958aaa8690c468da2acec47b2bb5f23'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.565266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1304/2618] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:24 2022] DELETE /identity/v3/users/9958aaa8690c468da2acec47b2bb5f23 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:24.740492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:24.751810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f0eb-e947-4c87-a683-1243dcb0f835 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:24.821624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:24.822496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:24.822783 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:24.823089 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] PATH_INFO: `/v3/projects/2007395425194750a87edd6c0da44e66` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:24.824563 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2007395425194750a87edd6c0da44e66)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:24.826484 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:24.874910 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2007395425194750a87edd6c0da44e66', 'request_id': 'req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.882717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2007395425194750a87edd6c0da44e66', 'request_id': 'req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.889334 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2007395425194750a87edd6c0da44e66', 'request_id': 'req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.900265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] The token cache is being invalidate because project 2007395425194750a87edd6c0da44e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:24.900588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.901237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc4a8abe-3a19-47d6-9cff-5830cb84ac30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:24.902809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1315/2619] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:24 2022] DELETE /identity/v3/projects/2007395425194750a87edd6c0da44e66 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:24.907855 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:24.908154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:24.908497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:24.928229 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:24.929669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:24.997636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1305/2620] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:25.006278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.006599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.006887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.028446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:25.029662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:25.044231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1316/2621] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:25.048536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.060630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43fe23-6bd5-4f0c-95c4-e2509ed0a4c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.068991 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.069823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.070118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.070397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.071157 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:54:25.071468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.073177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.086270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1306/2622] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:25.090633 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.100207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd5e1-d336-453d-8287-52321a7a1cf8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.105114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.105772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.106057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.106337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.107037 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.108993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.123173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:25.141423 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1317/2623] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:25 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:25.142728 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.149383 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:25.158152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d16be-3bf4-4ce7-9ad9-32a521f677ab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.215876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.216558 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.216844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.217120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.217806 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.219730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.244043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1307/2624] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:25 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:25.245202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.256460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ce46-2de2-4f31-bb13-51e0afca307d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.261099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.261757 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.262040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.262321 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.263043 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.264692 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.276119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1318/2625] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:25.280194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.290296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d688-b58d-44d2-807e-3c48ce619f0a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.294987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c113f1-e565-4d55-bde0-f479d2b037c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.295642 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c113f1-e565-4d55-bde0-f479d2b037c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.295933 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c113f1-e565-4d55-bde0-f479d2b037c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.296217 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c113f1-e565-4d55-bde0-f479d2b037c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.297123 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c113f1-e565-4d55-bde0-f479d2b037c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.298730 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c113f1-e565-4d55-bde0-f479d2b037c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.308711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1308/2626] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:25.312797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.324048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632a25-f313-4eb1-8f50-be61fb36eb99 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.328767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.329651 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.329935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.330217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] PATH_INFO: `/v3/projects/7dacce0444d148708e655e3ae6b4d94a/users/9f3865f086ac49b0affe1d7561ca21a1/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.347281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dacce0444d148708e655e3ae6b4d94a, user_id=9f3865f086ac49b0affe1d7561ca21a1, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.350115 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.363268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1319/2627] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:25 2022] PUT /identity/v3/projects/7dacce0444d148708e655e3ae6b4d94a/users/9f3865f086ac49b0affe1d7561ca21a1/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:25.374068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.374360 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.374646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.406719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] MFA Rules not processed for user `9f3865f086ac49b0affe1d7561ca21a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:25.407929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:25.477953 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1309/2628] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:25.489929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:25.491652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:25.575481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd30c0-69ef-4143-8fc4-87de8836290c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:25.581386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f1a44-9941-490c-8ca9-be8113793625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:25.582049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f1a44-9941-490c-8ca9-be8113793625 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:25.582338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f1a44-9941-490c-8ca9-be8113793625 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:25.582619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f1a44-9941-490c-8ca9-be8113793625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:25.590839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f1a44-9941-490c-8ca9-be8113793625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:25.592579 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f1a44-9941-490c-8ca9-be8113793625 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:25.601794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1320/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:33.908981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:33.972031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0802b6d-1586-40cb-9fee-db33b135ada8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:33.976688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:33.977563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:33.977850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:33.978132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] PATH_INFO: `/v3/users/9f3865f086ac49b0affe1d7561ca21a1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:33.979325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3865f086ac49b0affe1d7561ca21a1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:33.981305 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:33.994692 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:34.028688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3865f086ac49b0affe1d7561ca21a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.041712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3865f086ac49b0affe1d7561ca21a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.041712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3865f086ac49b0affe1d7561ca21a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.049015 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3865f086ac49b0affe1d7561ca21a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.049770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3865f086ac49b0affe1d7561ca21a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.062593 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1310/2630] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:33 2022] DELETE /identity/v3/users/9f3865f086ac49b0affe1d7561ca21a1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:34.067302 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.067771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.067869 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.085487 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:34.086706 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:34.145881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1321/2631] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:34.155700 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:34.157136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:34.217508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61a0ff-b477-46bc-a3bc-2b08f492d864 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:34.221105 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba8bc-9951-45cf-bfa6-93123d9408b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:34.221742 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba8bc-9951-45cf-bfa6-93123d9408b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.222021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba8bc-9951-45cf-bfa6-93123d9408b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.222498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba8bc-9951-45cf-bfa6-93123d9408b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.231428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba8bc-9951-45cf-bfa6-93123d9408b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:34.232912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba8bc-9951-45cf-bfa6-93123d9408b1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:34.242506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1311/2632] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:34.542039 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:34.543552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:34.558618 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab495643-71d8-4ed2-9a4b-cb5452eaaeb8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:34.564231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:34.565352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.565696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.565913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] PATH_INFO: `/v3/projects/7dacce0444d148708e655e3ae6b4d94a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.568118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dacce0444d148708e655e3ae6b4d94a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:34.570086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:34.615179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:34.621953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dacce0444d148708e655e3ae6b4d94a', 'request_id': 'req-11e9af13-3ddf-4a77-b382-242546c102e3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.630064 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dacce0444d148708e655e3ae6b4d94a', 'request_id': 'req-11e9af13-3ddf-4a77-b382-242546c102e3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.636515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dacce0444d148708e655e3ae6b4d94a', 'request_id': 'req-11e9af13-3ddf-4a77-b382-242546c102e3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.647382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] The token cache is being invalidate because project 7dacce0444d148708e655e3ae6b4d94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:34.647671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.648330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-11e9af13-3ddf-4a77-b382-242546c102e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:34.649894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1322/2633] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:34 2022] DELETE /identity/v3/projects/7dacce0444d148708e655e3ae6b4d94a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:34.654881 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.655171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.655453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.674039 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:34.675345 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:34.746804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1312/2634] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:34.754386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.754818 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.754980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.773777 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:34.774991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:34.783113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1323/2635] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:34.788003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:34.797824 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12215-5d3f-418e-9561-40e345ed589d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:34.802712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:34.803370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.803660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.803944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.804710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:54:34.805024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:34.806637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:34.819031 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1313/2636] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:34.823485 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:34.833843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c71208-7c8a-462b-a0d2-535f0747fab7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:34.838636 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:34.839305 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.839591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.839872 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.840613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:34.842580 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:34.869106 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1324/2637] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:34 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:54:34.873369 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:34.884030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef55ba-bfac-4167-87a5-a7fa67704cc3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:34.901799 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:34.944484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:34.945164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.945452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.945735 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.946422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:34.948387 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:34.969889 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1314/2638] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:34 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:54:34.974771 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:34.985028 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45be74c-472c-4de5-aa7e-50a2629ccb12 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:34.990498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:34.991157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:34.991441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:34.991717 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:34.992496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:34.994093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:35.004675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1325/2639] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:35.008737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:35.018772 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ffa5f-4de7-4cbb-a860-93c68cc47010 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:35.023681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cdcdc-9a21-4465-9109-e6a04dbeeebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:35.024362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cdcdc-9a21-4465-9109-e6a04dbeeebd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:35.024653 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cdcdc-9a21-4465-9109-e6a04dbeeebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:35.024978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cdcdc-9a21-4465-9109-e6a04dbeeebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:35.025829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cdcdc-9a21-4465-9109-e6a04dbeeebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:35.027768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cdcdc-9a21-4465-9109-e6a04dbeeebd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:35.038648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1315/2640] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:35.042829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:35.054018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab501c-2a8f-4afc-8061-6e8df8b03096 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:35.058765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:35.059648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:35.059936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:35.060214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] PATH_INFO: `/v3/projects/c02c9d962eec4da098d7eaf575a68625/users/3c0a9844a7c949959b9b013ee84b79ff/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:35.075238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02c9d962eec4da098d7eaf575a68625, user_id=3c0a9844a7c949959b9b013ee84b79ff, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:35.077945 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:35.091794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1326/2641] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:35 2022] PUT /identity/v3/projects/c02c9d962eec4da098d7eaf575a68625/users/3c0a9844a7c949959b9b013ee84b79ff/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:35.103103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8062c8-304c-444c-a982-891cb526274b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:35.103394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8062c8-304c-444c-a982-891cb526274b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:35.103679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8062c8-304c-444c-a982-891cb526274b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:35.131293 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-be8062c8-304c-444c-a982-891cb526274b None None] MFA Rules not processed for user `3c0a9844a7c949959b9b013ee84b79ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:35.132518 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be8062c8-304c-444c-a982-891cb526274b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:35.197813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1316/2642] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:35.209511 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:35.211286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:35.268333 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eb6b5-a181-4880-b4a5-b85e10f2cc78 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:35.272850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0a8b-b281-4524-8016-bca311df7b73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:35.273501 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0a8b-b281-4524-8016-bca311df7b73 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:35.273785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0a8b-b281-4524-8016-bca311df7b73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:35.274066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0a8b-b281-4524-8016-bca311df7b73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:35.282433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0a8b-b281-4524-8016-bca311df7b73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:35.283877 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0a8b-b281-4524-8016-bca311df7b73 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:35.293428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1327/2643] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:43.732106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8062c8-304c-444c-a982-891cb526274b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:43.812627 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8062c8-304c-444c-a982-891cb526274b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:43.818591 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:43.819514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:43.819798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:43.820078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] PATH_INFO: `/v3/users/3c0a9844a7c949959b9b013ee84b79ff` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:43.821605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0a9844a7c949959b9b013ee84b79ff)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:43.823796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:43.866386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:43.882655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0a9844a7c949959b9b013ee84b79ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:43.899313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0a9844a7c949959b9b013ee84b79ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:43.900121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0a9844a7c949959b9b013ee84b79ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:43.909576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0a9844a7c949959b9b013ee84b79ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:43.910722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0a9844a7c949959b9b013ee84b79ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:43.930034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1317/2644] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:43 2022] DELETE /identity/v3/users/3c0a9844a7c949959b9b013ee84b79ff => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:43.934590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:43.935127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:43.935491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:43.956035 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:43.958254 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:44.028337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1328/2645] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:44.038172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.039721 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:44.106916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cfada-afe2-4996-a3c9-c243aae114f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.110814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18da6-7a9a-44d2-9cc6-24769a58cb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.111480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18da6-7a9a-44d2-9cc6-24769a58cb38 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.111763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18da6-7a9a-44d2-9cc6-24769a58cb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.112036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18da6-7a9a-44d2-9cc6-24769a58cb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.122993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18da6-7a9a-44d2-9cc6-24769a58cb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.124553 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18da6-7a9a-44d2-9cc6-24769a58cb38 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.137437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1318/2646] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:44.455566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.457192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:44.470432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825a371-46a3-4203-8ea5-a59545685a1c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.475979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.476941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.477249 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.477547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] PATH_INFO: `/v3/projects/c02c9d962eec4da098d7eaf575a68625` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.479387 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02c9d962eec4da098d7eaf575a68625)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.483007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.486351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:44.542059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02c9d962eec4da098d7eaf575a68625', 'request_id': 'req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:44.549971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02c9d962eec4da098d7eaf575a68625', 'request_id': 'req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:44.557353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02c9d962eec4da098d7eaf575a68625', 'request_id': 'req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:44.568009 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] The token cache is being invalidate because project c02c9d962eec4da098d7eaf575a68625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:44.568203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:44.568891 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a8ef04ec-a7c1-4872-9451-e864f53cb6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:44.570617 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1329/2647] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:44 2022] DELETE /identity/v3/projects/c02c9d962eec4da098d7eaf575a68625 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:44.575835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.576121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.576432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.594364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:44.595557 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:44.658892 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1319/2648] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:44.667554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.667839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.668122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.688372 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:44.689592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:44.698031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1330/2649] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:54:44.702908 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.713084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170215a-1632-4cb0-a5a0-c5df43ede325 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.717917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.718616 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.718905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.719190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.719929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:54:44.720263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.721928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.735480 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1320/2650] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:44.741087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.750535 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc983758-fae6-4521-b3f5-0111f45166a8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.755827 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.756665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.756805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.757086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.757805 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.759736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.789160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1331/2651] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:44 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:44.793969 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.805190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a8cf-fc28-4e75-b4a7-14b3947c8a98 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.841714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:44.870122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.870853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.871146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.871432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.872214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.874471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.897549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1321/2652] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:44 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:44.902389 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.917392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656322c-be98-49ea-adc2-b7250cbdb1ea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.922671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.923364 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.923652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.923942 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.924758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.926567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.939007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1332/2653] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:44.943197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.954093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc8b8-5966-4859-b2e3-70ed3d8c457c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.960186 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1951f0-5a7c-4784-ad13-e3d985a4f69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:44.961118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1951f0-5a7c-4784-ad13-e3d985a4f69d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:44.961568 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1951f0-5a7c-4784-ad13-e3d985a4f69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:44.961983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1951f0-5a7c-4784-ad13-e3d985a4f69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:44.962868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1951f0-5a7c-4784-ad13-e3d985a4f69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:44.965125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1951f0-5a7c-4784-ad13-e3d985a4f69d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:44.979178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1322/2654] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:54:44.983599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:44.994324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763160-c63a-4206-87ac-f1fd976e6b91 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:44.999851 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.000806 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.001094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.001379 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] PATH_INFO: `/v3/projects/3ae54adab056452cb77df3a07659d67d/users/d6079f4e3db9445f8666d5fcc7b0c4f9/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.017493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae54adab056452cb77df3a07659d67d, user_id=d6079f4e3db9445f8666d5fcc7b0c4f9, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.020736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.035575 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1333/2655] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:44 2022] PUT /identity/v3/projects/3ae54adab056452cb77df3a07659d67d/users/d6079f4e3db9445f8666d5fcc7b0c4f9/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:45.047549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.047549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.047549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.076533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] MFA Rules not processed for user `d6079f4e3db9445f8666d5fcc7b0c4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:45.077826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:45.152682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1323/2656] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:45.157322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.228693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3338680-1de3-49f4-82e6-efca0861f348 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.233634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.234312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.234602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.234883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.235587 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.237552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.264661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1334/2657] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:45 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:45.269618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.282203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fce0d9-6927-4593-90cc-d13d339fc25d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.348896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.349586 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.349875 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.350159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.350872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.352939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.374574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1324/2658] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:45 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:45.383597 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.395512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bed001-b021-42b3-8ee1-3c46bda2adba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.400596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.401290 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.401607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.401917 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.402870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.404809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.416839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1335/2659] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:45.421194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.431463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90487ccc-421f-4c32-ad8b-218b44fb86ea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.436165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.437082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.437376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.437660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] PATH_INFO: `/v3/projects/136d75046e8e46758507f99a409145c9/users/9096b0a2743c44f1a304132c9156b4a7/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.454815 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=136d75046e8e46758507f99a409145c9, user_id=9096b0a2743c44f1a304132c9156b4a7, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.458212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.472947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1325/2660] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:45 2022] PUT /identity/v3/projects/136d75046e8e46758507f99a409145c9/users/9096b0a2743c44f1a304132c9156b4a7/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:45.477128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.484991 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:45.491668 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a7353-2bdb-4cf2-87ed-d7376bd0a60a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.571852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1afb4-cd1a-42cf-ab55-b4dc92584a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.572947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1afb4-cd1a-42cf-ab55-b4dc92584a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.573380 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1afb4-cd1a-42cf-ab55-b4dc92584a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.573802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1afb4-cd1a-42cf-ab55-b4dc92584a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.575241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1afb4-cd1a-42cf-ab55-b4dc92584a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.577799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1afb4-cd1a-42cf-ab55-b4dc92584a16 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.592939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1336/2661] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:45.597182 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.608113 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64377fcc-e13e-4b13-a339-3c465d6c8b5b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.613225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.614342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.614628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.614911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] PATH_INFO: `/v3/projects/136d75046e8e46758507f99a409145c9/users/9096b0a2743c44f1a304132c9156b4a7/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.617120 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=136d75046e8e46758507f99a409145c9, user_id=9096b0a2743c44f1a304132c9156b4a7, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.619821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.626314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:45.646358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1326/2662] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:45 2022] PUT /identity/v3/projects/136d75046e8e46758507f99a409145c9/users/9096b0a2743c44f1a304132c9156b4a7/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:45.658273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.658472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.658761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.694989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] MFA Rules not processed for user `9096b0a2743c44f1a304132c9156b4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:45.696289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:45.774235 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1337/2663] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:45.790958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:45.793092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:45.861412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a3b6-8544-48d1-93cc-13a2e32b9bdc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:45.865803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c14abb-44ce-4897-aa85-a2054e4cf919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:45.866492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c14abb-44ce-4897-aa85-a2054e4cf919 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:45.866798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c14abb-44ce-4897-aa85-a2054e4cf919 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:45.867064 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c14abb-44ce-4897-aa85-a2054e4cf919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:45.877581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c14abb-44ce-4897-aa85-a2054e4cf919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:45.879712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c14abb-44ce-4897-aa85-a2054e4cf919 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:45.891417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1327/2664] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:46.351191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:46.422703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac0a51-b72d-4086-8462-002021ec8dc5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:46.427956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:46.428882 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:46.429173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:46.429454 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] PATH_INFO: `/v3/users/d6079f4e3db9445f8666d5fcc7b0c4f9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:46.430716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6079f4e3db9445f8666d5fcc7b0c4f9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:46.432735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:46.464912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:46.488755 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6079f4e3db9445f8666d5fcc7b0c4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:46.489498 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6079f4e3db9445f8666d5fcc7b0c4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:46.497589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6079f4e3db9445f8666d5fcc7b0c4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:46.498314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6079f4e3db9445f8666d5fcc7b0c4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:46.507865 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6079f4e3db9445f8666d5fcc7b0c4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:46.528615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1338/2665] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:46 2022] DELETE /identity/v3/users/d6079f4e3db9445f8666d5fcc7b0c4f9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:46.533159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:46.533456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:46.533740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:46.552288 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:46.553533 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:46.617483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1328/2666] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:46.627090 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:46.628555 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:46.689194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68715e7-2c28-452f-8b49-94e912468b49 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:46.693160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:46.693820 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:46.694110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:46.694394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:46.703760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:46.705228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:46.715238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1339/2667] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:46.988563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:46.990097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:47.004212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c97a5-a6de-4adb-8e81-75c64975b24c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:47.008823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:47.009688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.009974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.010252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] PATH_INFO: `/v3/projects/3ae54adab056452cb77df3a07659d67d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.011947 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae54adab056452cb77df3a07659d67d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:47.014643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:47.062974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae54adab056452cb77df3a07659d67d', 'request_id': 'req-94d8b610-72ef-40d2-98d2-e7285a9911aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.068594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae54adab056452cb77df3a07659d67d', 'request_id': 'req-94d8b610-72ef-40d2-98d2-e7285a9911aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.076625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae54adab056452cb77df3a07659d67d', 'request_id': 'req-94d8b610-72ef-40d2-98d2-e7285a9911aa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.087950 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] The token cache is being invalidate because project 3ae54adab056452cb77df3a07659d67d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:47.088225 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.088980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.090509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1329/2668] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:46 2022] DELETE /identity/v3/projects/3ae54adab056452cb77df3a07659d67d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:47.094464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:47.104934 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04ba51-fbcd-4036-82e6-24dd375fc944 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:47.162760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:47.163627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.163915 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.164200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] PATH_INFO: `/v3/users/9096b0a2743c44f1a304132c9156b4a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.165433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9096b0a2743c44f1a304132c9156b4a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:47.167365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:47.213939 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9096b0a2743c44f1a304132c9156b4a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.214594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9096b0a2743c44f1a304132c9156b4a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.215850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:47.224189 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9096b0a2743c44f1a304132c9156b4a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.224956 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9096b0a2743c44f1a304132c9156b4a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.232327 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4fbacb9d-2d5d-49a2-8080-d3dc13c50cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9096b0a2743c44f1a304132c9156b4a7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.249825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1340/2669] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:47 2022] DELETE /identity/v3/users/9096b0a2743c44f1a304132c9156b4a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:47.533589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:47.535160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:47.549714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b610-72ef-40d2-98d2-e7285a9911aa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:47.575864 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:47.615067 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:47.615978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.616261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.616571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] PATH_INFO: `/v3/projects/136d75046e8e46758507f99a409145c9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.617793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=136d75046e8e46758507f99a409145c9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:47.619708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:47.665938 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136d75046e8e46758507f99a409145c9', 'request_id': 'req-34a12ac6-af3f-475f-b9aa-c64e055b36e0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.671506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136d75046e8e46758507f99a409145c9', 'request_id': 'req-34a12ac6-af3f-475f-b9aa-c64e055b36e0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.686321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136d75046e8e46758507f99a409145c9', 'request_id': 'req-34a12ac6-af3f-475f-b9aa-c64e055b36e0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.699230 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] The token cache is being invalidate because project 136d75046e8e46758507f99a409145c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:47.699777 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.700453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-34a12ac6-af3f-475f-b9aa-c64e055b36e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:47.702019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1330/2670] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:47 2022] DELETE /identity/v3/projects/136d75046e8e46758507f99a409145c9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:54:47.707205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.707747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.707747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.727586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:47.728839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:47.800726 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1341/2671] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:47.808519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.808813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.809102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.828448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:47.829672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:47.838346 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1331/2672] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:47.842684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:47.852576 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341c78d-1d24-4e54-af1c-7c1631b17942 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:47.857386 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:47.858048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.858340 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.858621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.859348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:54:47.859658 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:47.861295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:47.873318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1342/2673] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:47.877697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:47.890289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b649b-ffa1-43b1-8e13-0effc82fbb6c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:47.896008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:47.896717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:47.897005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:47.897289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:47.897983 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:47.900146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:47.928500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1332/2674] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:47 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:47.932800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:47.943654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31488e81-e667-4180-9f42-dce0fe062877 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.000178 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.000889 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.001180 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.001464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.002157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.004983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.025650 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1343/2675] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:47 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:48.029219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.039563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c3fcd-8441-4f84-b557-22100219a74d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.044131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.044827 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.045108 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.045389 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.046109 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.047719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.058706 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1333/2676] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:48.062544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.073057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05d577-0489-4336-b31c-26bc8935e0fa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.078258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4165a-733d-4412-ada8-03ce1dd9caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.079027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4165a-733d-4412-ada8-03ce1dd9caf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.079205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4165a-733d-4412-ada8-03ce1dd9caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.079525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4165a-733d-4412-ada8-03ce1dd9caf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.080512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4165a-733d-4412-ada8-03ce1dd9caf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.082236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4165a-733d-4412-ada8-03ce1dd9caf2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.092343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1344/2677] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:48.096450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.106425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56403c0-8d6e-4155-881a-3ba105d14cc2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.110989 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.111867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.112153 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.112477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] PATH_INFO: `/v3/projects/aa8b463380684449823e2d58cc233c27/users/e8984a755d154957934d07290962c591/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.127184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8b463380684449823e2d58cc233c27, user_id=e8984a755d154957934d07290962c591, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.130789 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.143846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1334/2678] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:48 2022] PUT /identity/v3/projects/aa8b463380684449823e2d58cc233c27/users/e8984a755d154957934d07290962c591/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:48.154383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.154673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.154951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.181790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] MFA Rules not processed for user `e8984a755d154957934d07290962c591`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:48.183000 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:48.202414 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:48.250250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1345/2679] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:48.255321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.309329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:48.321922 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadd4bd-262f-4848-99d2-ebca24f79af1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.327129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.327785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.328095 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.328396 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.329082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.330998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.359873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1335/2680] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:48 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:48.364895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.375470 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca4aa5-eb61-42f4-bfdf-b9faed577495 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.433188 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.433844 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.434122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.434394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.435075 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.437021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.457080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1346/2681] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:48 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:48.460961 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.471206 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53631d-96e0-4605-9bd8-57e60103a7ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.476497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.477152 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.477434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.477712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.478429 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.480106 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.492748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1336/2682] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:48.497400 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.510363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2320eb-1ab4-4714-9e2d-abb9ec266d26 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.515482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.516444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.516732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.517013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] PATH_INFO: `/v3/projects/3350806fef3544d688bf08b6689f93bd/users/e6e5b8d53bd0444790b33bb4a56a08ef/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.533419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3350806fef3544d688bf08b6689f93bd, user_id=e6e5b8d53bd0444790b33bb4a56a08ef, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.536754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.550199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1347/2683] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:48 2022] PUT /identity/v3/projects/3350806fef3544d688bf08b6689f93bd/users/e6e5b8d53bd0444790b33bb4a56a08ef/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:48.555117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.565716 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362f490-27c4-4807-82b3-b70fbf6a1f7e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.634740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8112065-7be6-4dd1-9123-4411a86ffb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.635418 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112065-7be6-4dd1-9123-4411a86ffb6f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.635709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112065-7be6-4dd1-9123-4411a86ffb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.635994 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8112065-7be6-4dd1-9123-4411a86ffb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.636769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8112065-7be6-4dd1-9123-4411a86ffb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.638397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8112065-7be6-4dd1-9123-4411a86ffb6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.649878 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1337/2684] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:48.654424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.670194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e82e70-323f-4979-8215-24ca5f74aaaa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.675311 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.677070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.677476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.677784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] PATH_INFO: `/v3/projects/3350806fef3544d688bf08b6689f93bd/users/e6e5b8d53bd0444790b33bb4a56a08ef/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.680021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3350806fef3544d688bf08b6689f93bd, user_id=e6e5b8d53bd0444790b33bb4a56a08ef, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.682837 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.694932 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1348/2685] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:48 2022] PUT /identity/v3/projects/3350806fef3544d688bf08b6689f93bd/users/e6e5b8d53bd0444790b33bb4a56a08ef/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:48.708913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.708913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.708913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.737116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] MFA Rules not processed for user `e6e5b8d53bd0444790b33bb4a56a08ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:48.738352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:48.814923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1338/2686] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:48.826307 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:48.828159 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:48.890199 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591deaf-feb6-480f-919c-377694553d62 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:48.893848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d3d6a-c52a-4856-8b82-231add05daab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:48.895104 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d3d6a-c52a-4856-8b82-231add05daab service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:48.895402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d3d6a-c52a-4856-8b82-231add05daab service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:48.895779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d3d6a-c52a-4856-8b82-231add05daab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:48.906032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d3d6a-c52a-4856-8b82-231add05daab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:48.907972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d3d6a-c52a-4856-8b82-231add05daab service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:48.921240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1349/2687] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:50.412437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:50.467669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:50.491764 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20eea4-26e3-4805-9eb1-53399baf3cd8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:50.497325 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:50.499301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:50.499301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:50.499301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] PATH_INFO: `/v3/users/e8984a755d154957934d07290962c591` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:50.500785 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8984a755d154957934d07290962c591)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:50.502905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:50.557237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8984a755d154957934d07290962c591'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:50.569816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8984a755d154957934d07290962c591'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:50.570511 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8984a755d154957934d07290962c591'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:50.579309 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8984a755d154957934d07290962c591'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:50.579987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8984a755d154957934d07290962c591'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:50.595025 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1339/2688] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:50 2022] DELETE /identity/v3/users/e8984a755d154957934d07290962c591 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:50.600128 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:50.600687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:50.600687 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:50.619237 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:50.620493 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:50.630699 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:50.690016 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1350/2689] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:50.700956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:50.702520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:50.763924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ca132-2f32-4ed5-954f-9793ff4355ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:50.767598 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:50.768257 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:50.768576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:50.768860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:50.778967 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:50.780451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:50.790704 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1340/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:51.084156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:51.085861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:51.103897 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166edb69-042d-4b0e-91b1-2fe5e025b3d0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:51.108879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:51.109845 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:51.110154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:51.110466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] PATH_INFO: `/v3/projects/aa8b463380684449823e2d58cc233c27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:51.112105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8b463380684449823e2d58cc233c27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:51.114406 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:51.168117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8b463380684449823e2d58cc233c27', 'request_id': 'req-6a883fab-ef24-430f-bc57-2a4475cc7f68'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.177268 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8b463380684449823e2d58cc233c27', 'request_id': 'req-6a883fab-ef24-430f-bc57-2a4475cc7f68'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.185995 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8b463380684449823e2d58cc233c27', 'request_id': 'req-6a883fab-ef24-430f-bc57-2a4475cc7f68'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.197514 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] The token cache is being invalidate because project aa8b463380684449823e2d58cc233c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:51.197800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.198609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.200566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1351/2691] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:51 2022] DELETE /identity/v3/projects/aa8b463380684449823e2d58cc233c27 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:51.205068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:51.215831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040c140-e6c2-4d9a-a48b-1dbde0d4e84b service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:51.274085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:51.274981 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:51.275261 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:51.275541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] PATH_INFO: `/v3/users/e6e5b8d53bd0444790b33bb4a56a08ef` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:51.276778 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e5b8d53bd0444790b33bb4a56a08ef)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:51.278711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:51.289822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:51.328276 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e5b8d53bd0444790b33bb4a56a08ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.341756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e5b8d53bd0444790b33bb4a56a08ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.342495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e5b8d53bd0444790b33bb4a56a08ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.350699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e5b8d53bd0444790b33bb4a56a08ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.351381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-60960fba-b7c0-4861-8f8c-01e0db1b661b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e5b8d53bd0444790b33bb4a56a08ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.364555 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1341/2692] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:51 2022] DELETE /identity/v3/users/e6e5b8d53bd0444790b33bb4a56a08ef => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:51.679087 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:51.682020 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:51.696368 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a883fab-ef24-430f-bc57-2a4475cc7f68 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:51.759226 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:51.760099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:51.760419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:51.760705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] PATH_INFO: `/v3/projects/3350806fef3544d688bf08b6689f93bd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:51.762156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3350806fef3544d688bf08b6689f93bd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:51.764130 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:51.809241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:51.817283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3350806fef3544d688bf08b6689f93bd', 'request_id': 'req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.826124 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3350806fef3544d688bf08b6689f93bd', 'request_id': 'req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.834350 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3350806fef3544d688bf08b6689f93bd', 'request_id': 'req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.846150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] The token cache is being invalidate because project 3350806fef3544d688bf08b6689f93bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:51.846439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.847120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c6dcf8-f3c3-4a4d-bffb-c8dcc1d6bf91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:51.849323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1352/2693] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:51 2022] DELETE /identity/v3/projects/3350806fef3544d688bf08b6689f93bd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:51.854413 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:51.854945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:51.855045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:51.884646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:51.886329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:51.959168 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1342/2694] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:51.967355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:51.967697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:51.967981 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:51.990637 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:51.991873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:52.000161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1353/2695] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:52.004823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.014847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde38f0-9cb3-4449-ae45-2a3ec6208e62 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.020103 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.020877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.021159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.021440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.022180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:54:52.022493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.024136 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.037907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1343/2696] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:52.042491 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.052679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3a485-bbb3-4227-8b74-b6d69b368a17 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.057529 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.058208 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.058502 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.058793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.059506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.061521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.093967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1354/2697] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:52 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:52.098804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.110683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dcabb-3391-4d1f-a702-7dbdcf1056a6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.178004 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.178696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.178972 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.179254 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.179963 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.181948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.204495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1344/2698] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:52 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:54:52.209776 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.226885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae1d0-bc0c-4e61-8341-e6bf21d1e839 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.235029 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.236221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.236778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.237289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.238674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.241974 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.264132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1355/2699] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:52.268563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.276300 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:52.283790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221938de-4b66-48f4-a78e-b926c131cdd7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.289871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25330b8e-8d21-459f-afb1-f8e5566676cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.290570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25330b8e-8d21-459f-afb1-f8e5566676cc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.290866 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25330b8e-8d21-459f-afb1-f8e5566676cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.291401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25330b8e-8d21-459f-afb1-f8e5566676cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.292367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25330b8e-8d21-459f-afb1-f8e5566676cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.294658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25330b8e-8d21-459f-afb1-f8e5566676cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.306052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1345/2700] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:54:52.311078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.322861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa35b49-dbc5-4e5c-bcc3-429535aaac16 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.328122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.329063 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.329351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.329631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] PATH_INFO: `/v3/projects/0fbe22062d9e47fa84b0439a03b8e505/users/1b37973525f14f9ebdea441487fad452/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.347481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbe22062d9e47fa84b0439a03b8e505, user_id=1b37973525f14f9ebdea441487fad452, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.350253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.367466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1356/2701] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:52 2022] PUT /identity/v3/projects/0fbe22062d9e47fa84b0439a03b8e505/users/1b37973525f14f9ebdea441487fad452/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:52.378400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.379490 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.379948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.411367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] MFA Rules not processed for user `1b37973525f14f9ebdea441487fad452`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:52.412664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:52.498535 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1346/2702] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:52.512160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.514263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:52.587023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fced05-02e6-4912-b609-db6890b19fc8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.591215 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23b14e-77be-418f-a27c-0b88ee55fea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.591997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23b14e-77be-418f-a27c-0b88ee55fea4 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.592469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23b14e-77be-418f-a27c-0b88ee55fea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.592830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23b14e-77be-418f-a27c-0b88ee55fea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.602448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23b14e-77be-418f-a27c-0b88ee55fea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.603920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23b14e-77be-418f-a27c-0b88ee55fea4 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.614444 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1357/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:52.634689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.703277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59de0c4-da10-45f5-be0d-cc64362f191f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:52.708698 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:52.709594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.709870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.710145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] PATH_INFO: `/v3/users/1b37973525f14f9ebdea441487fad452` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.711393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b37973525f14f9ebdea441487fad452)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:52.713398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:52.771603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b37973525f14f9ebdea441487fad452'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:52.790087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b37973525f14f9ebdea441487fad452'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:52.791053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b37973525f14f9ebdea441487fad452'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:52.801722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b37973525f14f9ebdea441487fad452'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:52.802414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b37973525f14f9ebdea441487fad452'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:52.817231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1347/2704] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:52 2022] DELETE /identity/v3/users/1b37973525f14f9ebdea441487fad452 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:52.822031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:52.822870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:52.822870 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:52.842410 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:52.843671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:52.920669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1358/2705] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:54:52.931221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:52.932792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:52.938352 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:52.999719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75211b-3010-464b-a1ef-f399b15475bf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.003647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f193f-4fe4-4dcd-a399-3d9c47c67241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.004353 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f193f-4fe4-4dcd-a399-3d9c47c67241 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.004650 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f193f-4fe4-4dcd-a399-3d9c47c67241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.005058 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f193f-4fe4-4dcd-a399-3d9c47c67241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.022196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f193f-4fe4-4dcd-a399-3d9c47c67241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.023597 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f193f-4fe4-4dcd-a399-3d9c47c67241 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:53.035782 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1348/2706] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:53.395948 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:53.398433 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:53.413474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7bb2-a2df-4690-a5ec-8adc01aa190e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.418822 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.419758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.420049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.420344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] PATH_INFO: `/v3/projects/0fbe22062d9e47fa84b0439a03b8e505` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.422506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fbe22062d9e47fa84b0439a03b8e505)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.424880 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:53.428070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:53.482565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fbe22062d9e47fa84b0439a03b8e505', 'request_id': 'req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:53.490701 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fbe22062d9e47fa84b0439a03b8e505', 'request_id': 'req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:53.498071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fbe22062d9e47fa84b0439a03b8e505', 'request_id': 'req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:53.509767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] The token cache is being invalidate because project 0fbe22062d9e47fa84b0439a03b8e505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:53.510057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:53.510716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-78310ca7-76ae-44d9-b1c4-ddab7bf057b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:53.512375 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1359/2707] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:53 2022] DELETE /identity/v3/projects/0fbe22062d9e47fa84b0439a03b8e505 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:53.517392 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.518051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.518437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.539302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:53.540647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:53.616223 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1349/2708] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:53.626538 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.627008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.627463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.649101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:53.650353 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:53.659306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1360/2709] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:53.666386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:53.677896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3b3d-43c4-418e-bd60-fd84d8aa3d29 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.682854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.683554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.683835 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.684117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.684903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:54:53.685214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.686858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:53.701292 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1350/2710] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 13:54:53.705850 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:53.716979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868892a-1e75-4de9-853b-2316bf9b8a4e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.722174 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.723968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.724427 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.724952 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.725747 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.727794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:53.761420 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1361/2711] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:53 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:53.767536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:53.786406 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fbdc6-b34c-4036-af9c-e4c3d3489514 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.880572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.881324 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.881652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.882032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.882790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.886008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:53.921115 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1351/2712] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:53 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:53.926217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:53.941542 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5b527-a2a8-4d0d-8c6e-8c3a46793c3b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.946768 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.947461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.947752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.948036 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.948908 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.950833 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:53.966660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1362/2713] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:53.970973 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:53.987860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab282fa8-f341-4a30-a67c-68a3f8e10e4c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:53.993644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818c31c-92ba-4085-bdaf-c9e2ea307353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:53.994388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c31c-92ba-4085-bdaf-c9e2ea307353 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:53.994714 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c31c-92ba-4085-bdaf-c9e2ea307353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:53.995065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818c31c-92ba-4085-bdaf-c9e2ea307353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:53.995931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818c31c-92ba-4085-bdaf-c9e2ea307353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:53.997805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818c31c-92ba-4085-bdaf-c9e2ea307353 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.009412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1352/2714] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:54.014019 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.025708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf4a2e-5223-43a0-83c0-02ed218ed498 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.030784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.031722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.032010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.032293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] PATH_INFO: `/v3/projects/74018b206f3d47ada9e0aef3a599be9d/users/11d917c2d523497b98e4de05e195fb79/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.049250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=74018b206f3d47ada9e0aef3a599be9d, user_id=11d917c2d523497b98e4de05e195fb79, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.051986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.068463 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1363/2715] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:54 2022] PUT /identity/v3/projects/74018b206f3d47ada9e0aef3a599be9d/users/11d917c2d523497b98e4de05e195fb79/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:54.080519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.080988 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.081455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.109715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:54.120638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] MFA Rules not processed for user `11d917c2d523497b98e4de05e195fb79`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:54.121890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:54.192860 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1353/2716] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:54.198593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.276152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ff489-0d76-42ee-a858-4c33b128c0ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.281046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.281840 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.282008 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.282283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.283024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.285623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.328317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1364/2717] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:54:54 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:54:54.333509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.345525 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142aeff-f761-40fd-8aba-d583aac1af67 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.419244 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.420085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.420516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.420909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.421731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.423873 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.447165 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1354/2718] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:54:54 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 13:54:54.451953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.463472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a9977-edb6-4a7b-8f7e-294d9b5b3f08 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.468559 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.469259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.469550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.469836 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.470593 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.472258 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.485399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1365/2719] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:54:54.490215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.501359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d18b2-6b41-434b-831b-a1229846f938 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.507559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.508530 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.508818 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.509096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] PATH_INFO: `/v3/projects/a5275cf0456c48da8dcd9ac4decfcb12/users/768a74f91aac483aa07fdf7524036f5a/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.526519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5275cf0456c48da8dcd9ac4decfcb12, user_id=768a74f91aac483aa07fdf7524036f5a, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.529295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.545070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1355/2720] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:54 2022] PUT /identity/v3/projects/a5275cf0456c48da8dcd9ac4decfcb12/users/768a74f91aac483aa07fdf7524036f5a/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:54.549167 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.555239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:54.561968 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09351b8-4e22-49b9-8c1f-2606e60ddadd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.630278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db782c5-b0a9-4199-9a8d-3684ccc5ff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.630973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db782c5-b0a9-4199-9a8d-3684ccc5ff3f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.631264 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db782c5-b0a9-4199-9a8d-3684ccc5ff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.631550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db782c5-b0a9-4199-9a8d-3684ccc5ff3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.632336 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db782c5-b0a9-4199-9a8d-3684ccc5ff3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.634076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db782c5-b0a9-4199-9a8d-3684ccc5ff3f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.646166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1366/2721] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:54:54.650272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.662189 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122964d1-ce37-40b8-a55e-4311a659fcd5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.667174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.668089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.668483 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.668768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] PATH_INFO: `/v3/projects/a5275cf0456c48da8dcd9ac4decfcb12/users/768a74f91aac483aa07fdf7524036f5a/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.670877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5275cf0456c48da8dcd9ac4decfcb12, user_id=768a74f91aac483aa07fdf7524036f5a, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:54.673652 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:54.687666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1356/2722] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:54:54 2022] PUT /identity/v3/projects/a5275cf0456c48da8dcd9ac4decfcb12/users/768a74f91aac483aa07fdf7524036f5a/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:54.700665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.700954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.701158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.733344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] MFA Rules not processed for user `768a74f91aac483aa07fdf7524036f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:54.734615 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:54.812957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1367/2723] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:54.824573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:54.826599 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:54.904637 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78f4f3-332a-4ffa-9130-a05b0081bca1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:54.911178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:54.911871 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:54.912156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:54.912475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:54.921466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:55.001279 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:55.004061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:55.022118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1357/2724] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:55.811362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:55.824219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657f335-6f83-40e9-a1b8-e89912f024aa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:55.828450 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:55.829133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:55.829424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:55.829708 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:55.831611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:55.924324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:55.926660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:55.936702 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:55.948219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1368/2725] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:56.308853 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:56.310527 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:56.382701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce455f1b-079f-4ef9-abed-8e7bb8b8d726 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:56.387856 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:56.388680 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:56.389077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:56.389451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:56.392070 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:56.474948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:56.479078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:56.488971 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1358/2726] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:54:57.628913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:57.694003 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:57.695741 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:57.709420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839b33a-3aef-4202-a4ea-0ea378213c8b service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:57.712788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb9b3d-798a-4ab6-b797-d0f081a908c1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:57.715774 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:57.717122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:57.717679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:57.717930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:57.718239 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:57.718613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:57.718898 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:57.719175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] PATH_INFO: `/v3/users/11d917c2d523497b98e4de05e195fb79` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:57.720656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d917c2d523497b98e4de05e195fb79)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:57.721773 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:57.723558 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:57.744416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:57.747205 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa0943-3dc2-4f69-8b2d-a1a0a66810db service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:57.769032 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1359/2727] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 13:54:57.779957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d917c2d523497b98e4de05e195fb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:57.781295 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d917c2d523497b98e4de05e195fb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:57.795071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d917c2d523497b98e4de05e195fb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:57.796198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d917c2d523497b98e4de05e195fb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:57.807722 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d917c2d523497b98e4de05e195fb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:57.830612 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1369/2728] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:57 2022] DELETE /identity/v3/users/11d917c2d523497b98e4de05e195fb79 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:57.836078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:57.836471 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:57.836699 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:57.857092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:54:57.858271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:57.907576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:57.944665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1360/2729] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:54:57.954802 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:57.956312 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:58.018816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823c69-43f6-45b5-a927-a83ecec1977e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:58.022745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:58.023408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:58.023689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:58.023972 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:58.033973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:58.036179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:58.046524 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1370/2730] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:54:58.538240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:58.539852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:58.572086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b396316-87f7-4798-acd0-06be5e5c0bf1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:58.578273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:58.579238 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:58.579615 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:58.579954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] PATH_INFO: `/v3/projects/74018b206f3d47ada9e0aef3a599be9d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:58.581376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] RBAC: Authorizing `identity:delete_project(project_id=74018b206f3d47ada9e0aef3a599be9d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:58.584112 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:58.634526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74018b206f3d47ada9e0aef3a599be9d', 'request_id': 'req-ab3916ba-d051-4700-afc4-ca7d9dac33df'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.641570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74018b206f3d47ada9e0aef3a599be9d', 'request_id': 'req-ab3916ba-d051-4700-afc4-ca7d9dac33df'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.651061 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74018b206f3d47ada9e0aef3a599be9d', 'request_id': 'req-ab3916ba-d051-4700-afc4-ca7d9dac33df'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.664930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] The token cache is being invalidate because project 74018b206f3d47ada9e0aef3a599be9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:58.665224 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.665904 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.667512 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1361/2731] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:58 2022] DELETE /identity/v3/projects/74018b206f3d47ada9e0aef3a599be9d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:58.671553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:58.681830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a262f6-d349-4d01-b6a3-82442bcf78a8 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:58.701448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:58.765517 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:58.766497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:58.766875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:58.767225 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] PATH_INFO: `/v3/users/768a74f91aac483aa07fdf7524036f5a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:58.768554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] RBAC: Authorizing `identity:delete_user(user_id=768a74f91aac483aa07fdf7524036f5a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:58.771357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:58.825205 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768a74f91aac483aa07fdf7524036f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.825903 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768a74f91aac483aa07fdf7524036f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.833782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768a74f91aac483aa07fdf7524036f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.834503 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768a74f91aac483aa07fdf7524036f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.845059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768a74f91aac483aa07fdf7524036f5a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:58.865875 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1371/2732] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:54:58 2022] DELETE /identity/v3/users/768a74f91aac483aa07fdf7524036f5a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:54:59.333199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:54:59.336495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:54:59.363668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3916ba-d051-4700-afc4-ca7d9dac33df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:54:59.451494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:54:59.452447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:54:59.452800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:54:59.453012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] PATH_INFO: `/v3/projects/a5275cf0456c48da8dcd9ac4decfcb12` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:54:59.454316 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5275cf0456c48da8dcd9ac4decfcb12)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:54:59.456266 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:54:59.516250 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5275cf0456c48da8dcd9ac4decfcb12', 'request_id': 'req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:59.525625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5275cf0456c48da8dcd9ac4decfcb12', 'request_id': 'req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:59.530063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:54:59.540588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5275cf0456c48da8dcd9ac4decfcb12', 'request_id': 'req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:59.560019 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] The token cache is being invalidate because project a5275cf0456c48da8dcd9ac4decfcb12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:54:59.560411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:59.561405 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:54:59.565813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1362/2733] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:54:59 2022] DELETE /identity/v3/projects/a5275cf0456c48da8dcd9ac4decfcb12 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:55:03.359047 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:03.360497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:03.421300 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee0c0b-365d-4bc4-8dcf-d4263e15f58b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:03.469594 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:03.470239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:03.470518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:03.470790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:03.473134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:03.477933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:03.543139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:03.544648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:03.555588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1372/2734] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:55:06.875049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:06.878580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:06.898872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12b3e-f65e-452b-abe5-611ac07cdeb8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:06.904847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:06.905862 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:06.906159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:06.906446 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] PATH_INFO: `/v3/users/e13f5cad16824554a1fb32ed9a915ae9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:06.908197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13f5cad16824554a1fb32ed9a915ae9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:06.911000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:06.967494 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13f5cad16824554a1fb32ed9a915ae9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:06.987460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13f5cad16824554a1fb32ed9a915ae9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:06.989134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13f5cad16824554a1fb32ed9a915ae9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:06.998133 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13f5cad16824554a1fb32ed9a915ae9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:06.999335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a681d73b-ecca-47c3-b64e-3480fe332184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13f5cad16824554a1fb32ed9a915ae9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:07.015198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1363/2735] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 13:55:06 2022] DELETE /identity/v3/users/e13f5cad16824554a1fb32ed9a915ae9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:55:07.214260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.225695 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5cccc-c66f-4a97-9208-cf430a737312 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.294941 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.295808 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.296096 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.296413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] PATH_INFO: `/v3/projects/1ce476b2a1764f56b9780f4a5c75ac3d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.297603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce476b2a1764f56b9780f4a5c75ac3d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.299581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.347079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce476b2a1764f56b9780f4a5c75ac3d', 'request_id': 'req-15c3e77b-af39-44b8-814b-99ff508d9b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:07.355239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce476b2a1764f56b9780f4a5c75ac3d', 'request_id': 'req-15c3e77b-af39-44b8-814b-99ff508d9b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:07.361839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce476b2a1764f56b9780f4a5c75ac3d', 'request_id': 'req-15c3e77b-af39-44b8-814b-99ff508d9b93'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:07.372528 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] The token cache is being invalidate because project 1ce476b2a1764f56b9780f4a5c75ac3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 13:55:07.372821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:07.373469 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-15c3e77b-af39-44b8-814b-99ff508d9b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 13:55:07.375002 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1373/2736] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 13:55:07 2022] DELETE /identity/v3/projects/1ce476b2a1764f56b9780f4a5c75ac3d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 13:55:07.381008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.381313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.381605 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.401451 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:07.402670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:07.463763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1364/2737] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:07.471770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.472061 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.472402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.491373 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:07.492605 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:07.500655 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1374/2738] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:07.505425 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.515245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b0aa-799d-4960-a632-ed9482fb5b48 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.519943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.520648 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.520939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.521221 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.521948 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 13:55:07.522259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.524005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.536209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1365/2739] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 13:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 13:55:07.540639 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.549960 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b451fc7-d495-4e22-aa7a-b01e31dcdf06 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.555269 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.555931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.556221 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.556536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.557239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.559152 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.585679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1375/2740] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 13:55:07 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:07.589572 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.600294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9cdc-e69a-4739-bba8-1459c64f380b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.645755 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:07.659528 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.660805 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.661197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.661485 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.662214 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.664156 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.685059 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1366/2741] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 13:55:07 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 13:55:07.689242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.695066 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:07.704292 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290cd0-ca5c-4884-a245-1f98a9517b30 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.710382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.711057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.711362 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.711627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.712383 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.714018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.725479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1376/2742] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 13:55:07.729589 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.739798 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2367944-3e3a-4dc6-acc4-c93a0fb3ce8d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.744584 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9573c-84ca-48b4-b30c-7589a5091455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.745290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9573c-84ca-48b4-b30c-7589a5091455 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.745578 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9573c-84ca-48b4-b30c-7589a5091455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.745863 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9573c-84ca-48b4-b30c-7589a5091455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.746643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9573c-84ca-48b4-b30c-7589a5091455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.748272 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9573c-84ca-48b4-b30c-7589a5091455 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.761297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1367/2743] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 13:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 13:55:07.765551 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.777059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b48b66-bc2a-4eb9-9128-89d4e889a2de admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.781788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.782674 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.782962 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.783246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] PATH_INFO: `/v3/projects/b596535d63d6490eb939c300bfe565cf/users/ae843854dccd4c28a260d040be460db0/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.799005 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b596535d63d6490eb939c300bfe565cf, user_id=ae843854dccd4c28a260d040be460db0, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.802200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:07.817719 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1377/2744] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 13:55:07 2022] PUT /identity/v3/projects/b596535d63d6490eb939c300bfe565cf/users/ae843854dccd4c28a260d040be460db0/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 13:55:07.823227 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d8c6-c951-4d34-adb5-b5e4d6d0e77a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.823526 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d8c6-c951-4d34-adb5-b5e4d6d0e77a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.823809 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d8c6-c951-4d34-adb5-b5e4d6d0e77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.842669 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b599d8c6-c951-4d34-adb5-b5e4d6d0e77a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:07.844010 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b599d8c6-c951-4d34-adb5-b5e4d6d0e77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:07.910949 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1368/2745] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:55:07.921760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:07.923603 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:07.981574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf757a9-ac68-48ed-9526-ab8b74284954 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:07.985714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:07.986360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:07.986644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:07.986923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:07.995992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:07.997465 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:08.007659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1378/2746] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:55:12.345903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f373c4-e2fc-4193-b27c-185198c7b4c5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:12.346394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f373c4-e2fc-4193-b27c-185198c7b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:12.346540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f373c4-e2fc-4193-b27c-185198c7b4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:12.378204 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-12f373c4-e2fc-4193-b27c-185198c7b4c5 None None] MFA Rules not processed for user `ae843854dccd4c28a260d040be460db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:12.379554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-12f373c4-e2fc-4193-b27c-185198c7b4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:12.455960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1369/2747] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:55:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:55:12.470086 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:12.471648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:12.570848 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37547bee-91c4-4bb1-94db-db220ba75291 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:12.578623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:12.579297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:12.579590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:12.579867 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:12.588894 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:12.590643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:12.602369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1379/2748] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:55:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:55:13.997940 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:13.998390 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:13.998496 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:14.017303 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] MFA Rules not processed for user `ae843854dccd4c28a260d040be460db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:55:14.018444 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:14.026155 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1370/2749] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:55:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 13:55:14.362094 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:14.364583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:14.376745 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0e93c-fa53-4440-942a-7e8a44ae19b0 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:14.381351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:14.382014 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:14.382296 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:14.382572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:14.384449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:14.394905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:14.396516 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:14.406472 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1380/2750] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:55:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:55:16.623851 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:55:16.636888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4d714-0272-48c0-a2f7-42510e36fe72 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:55:16.641634 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:55:16.642439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:55:16.642843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:55:16.643228 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:55:16.645410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:55:16.662311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:55:16.663802 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f04abd-e5e2-4c27-8a21-6b61833f36b7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:55:16.673131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1371/2751] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:55:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:55:24.402101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:55:26.669847 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:35.687673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:35.762011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1a015-0900-4fea-8a3a-9fc3eda0d3af service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:35.768585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:35.769607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:35.770040 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:35.770430 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:35.772693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:35.807967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:35.809732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:35.821685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1381/2752] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:56:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:36.628412 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:36.629011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:36.629011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:36.648464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] MFA Rules not processed for user `f98684788906448bb3b35af5a2fc017f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:36.649696 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:36.710145 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1372/2753] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:37.032582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.052736 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8117f-51ab-4155-b920-ce41b1f5c242 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.056275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.056950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.057241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.057525 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.059382 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:37.084185 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.085679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030b8fb-d2d5-4e5e-8911-fbbed656ce7f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.094031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1382/2754] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:56:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:37.575517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:37.577134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:37.653895 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad4db-6ac4-4550-8f9d-7ed52ff42840 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:37.658946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:37.659600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:37.660172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:37.660172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:37.662127 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:37.674203 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:37.675868 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:37.686916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1373/2755] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:56:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:42.312490 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:42.313145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:42.313145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:42.334455 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] MFA Rules not processed for user `ae843854dccd4c28a260d040be460db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:56:42.335763 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:42.348784 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1383/2756] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:56:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:56:47.682237 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=57 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:56:51.488985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:56:51.490461 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:51.506298 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a067d-9198-4c9d-98a8-67748f9e57b7 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:56:51.511832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:56:51.512520 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:56:51.512812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:56:51.513098 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:56:51.514909 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:56:51.528470 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:56:51.530101 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78879b-3146-4b84-a000-f097b3104f53 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:56:51.542036 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1374/2757] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:56:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:56:52.329727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=46 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:01.537343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:57:51.725384 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:57:51.726804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:51.767994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12f5b9-b48d-4212-bc6c-6323d2ef34a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:57:51.773613 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:57:51.774273 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:57:51.774561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:57:51.774843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:57:51.776947 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:57:51.791879 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:57:51.794935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:57:51.820160 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1384/2758] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:57:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:01.809765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=15 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:22.699467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701b50-4870-4f04-90a7-b6a97e00b615 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:22.700043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701b50-4870-4f04-90a7-b6a97e00b615 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:22.700043 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701b50-4870-4f04-90a7-b6a97e00b615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:22.719183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0a701b50-4870-4f04-90a7-b6a97e00b615 None None] MFA Rules not processed for user `ae843854dccd4c28a260d040be460db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:22.720435 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0a701b50-4870-4f04-90a7-b6a97e00b615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:22.728008 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1375/2759] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:58:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:32.713703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:58:47.986275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:58:47.987704 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:48.065775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a03d4f-2b50-4281-934e-78fc6b2031d3 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:58:48.069482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:58:48.070127 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:48.070409 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:48.070686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:48.072476 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:48.084365 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:58:48.085793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:58:48.094561 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1385/2760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:58:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:58:58.054359 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:58:58.054921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:58:58.054921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:58:58.072711 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] MFA Rules not processed for user `f98684788906448bb3b35af5a2fc017f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 13:58:58.073917 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:58:58.080923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1376/2761] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 13:58:58.090556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=35 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:08.067571 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:20.942746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:20.952120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b4cc3-5ba6-41ac-8ce7-683cb899004e service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:20.955823 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:20.956583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:20.956874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:20.957157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:20.959186 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:21.022381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:21.023775 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:21.034370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1386/2762] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:59:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 13:59:31.029412 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 13:59:59.816027 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 13:59:59.817610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:59.836828 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03499a2-b3dd-4d36-a80d-98d5de9cda2e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 13:59:59.842087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 13:59:59.842825 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 13:59:59.843235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 13:59:59.843619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 13:59:59.846980 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 13:59:59.885312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 13:59:59.886784 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d0bd2-47c7-4cbd-ae45-4ca96f981796 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 13:59:59.909012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1377/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 13:59:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:00.077553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:00.088402 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b69a-b407-481e-b04e-97b650e4e054 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:00.091986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:00.092675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:00.092966 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:00.093245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:00.095154 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:00.109780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:00.111162 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:00.123338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1387/2764] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:00:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:07.190313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:07.190902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:07.191088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:07.209839 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] MFA Rules not processed for user `ae843854dccd4c28a260d040be460db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:00:07.211075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:07.218438 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1378/2765] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:00:10.117645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:12.687663 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:12.698976 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7c8ee-43c9-41f9-8236-0019a7126687 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:12.704961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:12.705614 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:12.705896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:12.706171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:12.714363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:12.715933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:12.725281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1388/2766] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:00:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:16.572762 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:16.582554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c8392-32f8-41b0-bfcc-2e139172e2d3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:16.586393 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:16.587037 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:16.587326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:16.587609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:16.595259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:16.596916 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:16.606343 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1379/2767] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:00:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:19.742600 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:00:19.754351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d574e64-8b25-48b0-aa83-02e4a85257a9 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:00:19.760082 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:00:19.760933 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:00:19.761288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:00:19.761634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:00:19.763896 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:00:19.776518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:00:19.778213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:00:19.790011 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1389/2768] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:00:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:00:26.601816 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=27 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:00:29.785157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=12 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:24.739911 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:24.748658 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70957e-5bbb-49ec-8cef-66fd89654eb8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:24.752377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:24.753017 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:24.753301 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:24.753580 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:24.755388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:24.790426 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:24.791877 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:24.811146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1380/2769] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:27.692693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:27.716571 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d443f-edfb-4dee-a946-ac9c1198e44c service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:27.723914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:27.724936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:27.725405 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:27.725883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:27.727772 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:27.743652 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:27.746759 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:27.760417 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1390/2770] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:30.902012 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:30.925131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b332b-2efc-4c62-92f7-0b6e560a1373 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:30.930946 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:30.931609 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:30.931896 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:30.932180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:30.934104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:30.947198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:30.949088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:30.967166 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1381/2771] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:30.967680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:30.969149 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:30.983572 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc9a60-9133-44f7-acff-f57078be92be service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:30.989006 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:30.989911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:30.990203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:30.990486 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] PATH_INFO: `/v3/users/f98684788906448bb3b35af5a2fc017f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:30.991703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f98684788906448bb3b35af5a2fc017f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:30.993785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:31.042263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f98684788906448bb3b35af5a2fc017f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.042919 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f98684788906448bb3b35af5a2fc017f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.049994 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f98684788906448bb3b35af5a2fc017f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.050679 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f98684788906448bb3b35af5a2fc017f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.058351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f98684788906448bb3b35af5a2fc017f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.075769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1391/2772] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:01:30 2022] DELETE /identity/v3/users/f98684788906448bb3b35af5a2fc017f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:31.261388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:31.275057 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e656e-8fb9-433d-b942-be11c4e4d173 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:31.338536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:31.339433 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:31.339748 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:31.340029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] PATH_INFO: `/v3/projects/189885bf9a5944bf8eaedd89bb5bec9f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:31.341259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189885bf9a5944bf8eaedd89bb5bec9f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:31.343213 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:31.390378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189885bf9a5944bf8eaedd89bb5bec9f', 'request_id': 'req-a9baae6b-309b-4532-8872-d5f0db656ef9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.396821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189885bf9a5944bf8eaedd89bb5bec9f', 'request_id': 'req-a9baae6b-309b-4532-8872-d5f0db656ef9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.405275 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189885bf9a5944bf8eaedd89bb5bec9f', 'request_id': 'req-a9baae6b-309b-4532-8872-d5f0db656ef9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.417486 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] The token cache is being invalidate because project 189885bf9a5944bf8eaedd89bb5bec9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:31.417780 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.418432 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.419974 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1382/2773] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:01:31 2022] DELETE /identity/v3/projects/189885bf9a5944bf8eaedd89bb5bec9f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:31.424191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:31.425698 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:31.439236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e184e-0324-48bd-8e9e-f9138a9d0122 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:31.494929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:31.495798 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:31.496081 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:31.496381 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] PATH_INFO: `/v3/users/d6cbdab41f8644618d187cac794c8244` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:31.497586 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6cbdab41f8644618d187cac794c8244)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:31.499500 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:31.545439 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6cbdab41f8644618d187cac794c8244'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.546084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6cbdab41f8644618d187cac794c8244'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.553703 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6cbdab41f8644618d187cac794c8244'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.554355 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6cbdab41f8644618d187cac794c8244'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.557936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:31.562758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6cbdab41f8644618d187cac794c8244'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.581843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1392/2774] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:01:31 2022] DELETE /identity/v3/users/d6cbdab41f8644618d187cac794c8244 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:31.758149 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:31.759566 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:31.773164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baae6b-309b-4532-8872-d5f0db656ef9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:31.807311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:31.835638 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:31.836800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:31.837092 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:31.837381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] PATH_INFO: `/v3/projects/19c7ce863a774fd6bd3d3cd5d8767ac9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:31.838573 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c7ce863a774fd6bd3d3cd5d8767ac9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:31.840522 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:31.887296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c7ce863a774fd6bd3d3cd5d8767ac9', 'request_id': 'req-41767665-7ef3-4713-9e79-1a79627cf151'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.893173 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c7ce863a774fd6bd3d3cd5d8767ac9', 'request_id': 'req-41767665-7ef3-4713-9e79-1a79627cf151'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.901063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c7ce863a774fd6bd3d3cd5d8767ac9', 'request_id': 'req-41767665-7ef3-4713-9e79-1a79627cf151'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.912813 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] The token cache is being invalidate because project 19c7ce863a774fd6bd3d3cd5d8767ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:31.913462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.913978 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:31.915610 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1383/2775] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:01:31 2022] DELETE /identity/v3/projects/19c7ce863a774fd6bd3d3cd5d8767ac9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:41.578349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:41.909370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:47.444370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:47.446236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:47.555697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edae91-e6e9-4b4e-a18f-94d5e30d68ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:47.638223 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:47.638810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:47.639100 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:47.639441 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:47.642322 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:47.656133 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:47.658675 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:47.671048 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1393/2776] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:52.549710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:52.551154 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:52.608395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41767665-7ef3-4713-9e79-1a79627cf151 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:52.612670 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:52.613331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:52.613620 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:52.613905 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:52.616549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:52.682629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:52.684090 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:52.694419 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1384/2777] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:55.594809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:55.596466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:55.611967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f62171-b86e-4e00-a3b6-dbb518b68ed3 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:55.616656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:55.617543 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:55.617830 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:55.618109 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] PATH_INFO: `/v3/users/ae843854dccd4c28a260d040be460db0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:55.619360 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae843854dccd4c28a260d040be460db0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:55.621361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:55.677349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae843854dccd4c28a260d040be460db0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:55.678033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae843854dccd4c28a260d040be460db0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:55.685447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae843854dccd4c28a260d040be460db0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:55.686131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae843854dccd4c28a260d040be460db0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:55.697446 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59770b5-c107-4405-804e-396b7b77f328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae843854dccd4c28a260d040be460db0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:55.709661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:55.718067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1394/2778] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:01:55 2022] DELETE /identity/v3/users/ae843854dccd4c28a260d040be460db0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:55.895194 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:55.905118 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d146-1ed9-482b-b865-b71d1d5bb7d5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:55.973964 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:55.974768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:55.975051 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:55.975321 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] PATH_INFO: `/v3/projects/b596535d63d6490eb939c300bfe565cf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:55.976537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b596535d63d6490eb939c300bfe565cf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:55.978473 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.024554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b596535d63d6490eb939c300bfe565cf', 'request_id': 'req-21be266f-12a3-4866-bff1-8b954673b652'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:56.026539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:56.031076 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b596535d63d6490eb939c300bfe565cf', 'request_id': 'req-21be266f-12a3-4866-bff1-8b954673b652'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:56.039134 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b596535d63d6490eb939c300bfe565cf', 'request_id': 'req-21be266f-12a3-4866-bff1-8b954673b652'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:56.050790 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] The token cache is being invalidate because project b596535d63d6490eb939c300bfe565cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:56.051080 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:56.051715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21be266f-12a3-4866-bff1-8b954673b652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:56.053291 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1385/2779] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:01:55 2022] DELETE /identity/v3/projects/b596535d63d6490eb939c300bfe565cf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:56.058366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.058660 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.058977 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.115475 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:56.116659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:56.177831 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1395/2780] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:56.185477 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.185771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.186056 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.205611 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:56.206945 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:56.215125 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1386/2781] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:56.219957 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.229607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedb88b-1af8-495a-8b06-8937f9aa0431 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.234093 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.234743 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.235024 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.235301 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.236113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:01:56.236464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.238072 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.249474 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1396/2782] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:01:56.253800 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.263073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f367f6-89f1-4f65-8078-521d99eac11e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.268271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.268951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.269231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.269508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.270190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.272085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.299756 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1387/2783] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:56.302667 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.312761 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf1e9b-3901-469f-8282-4ff2de8ee701 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.398398 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.398990 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.399280 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.399567 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.400285 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.402257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.422281 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1397/2784] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:56.426644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.436921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf4445-cc52-4cbe-8040-0001d1ad6b3f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.442628 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.443302 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.443600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.443870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.444643 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.446617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.458000 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1388/2785] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:01:56.462795 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.473366 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef2de6-b904-4afc-a034-bf34d517554a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.478341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c195f02-440c-4607-902a-6dc85086f810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.479010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195f02-440c-4607-902a-6dc85086f810 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.479332 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195f02-440c-4607-902a-6dc85086f810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.479648 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c195f02-440c-4607-902a-6dc85086f810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.480505 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c195f02-440c-4607-902a-6dc85086f810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.482236 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c195f02-440c-4607-902a-6dc85086f810 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.492598 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1398/2786] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:56.496795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.507621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524dff5-1c72-4550-a0c9-2c545031e447 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.512489 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.513379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.513666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.513951 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] PATH_INFO: `/v3/projects/8b5d5aeec557494bb95ac3e10171c1c9/users/fd025e99948246979c0046a74207ac7d/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.529503 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5d5aeec557494bb95ac3e10171c1c9, user_id=fd025e99948246979c0046a74207ac7d, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.532252 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.546365 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1389/2787] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:01:56 2022] PUT /identity/v3/projects/8b5d5aeec557494bb95ac3e10171c1c9/users/fd025e99948246979c0046a74207ac7d/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:56.557442 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.557814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.558168 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.585929 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] MFA Rules not processed for user `fd025e99948246979c0046a74207ac7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:56.587416 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:56.605765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:56.659341 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1399/2788] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:56.663822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.745770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e73c3-eaf8-4a30-825c-0b8266e89a93 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.750465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.751122 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.751402 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.751717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.752840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.755854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.783928 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1390/2789] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:56.788101 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.798787 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59828cca-3fbe-4d22-8f7d-ef2d66a98c47 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.858871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.859588 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.859958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.860335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.861074 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.863140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.885645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1400/2790] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:01:56 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:56.887995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.898715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a32494-f121-42d1-9af9-810d9bd1066f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.903376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.904029 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.904326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.904619 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.905333 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.907844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.920659 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1391/2791] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:56.924432 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.935241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebb348-c5dc-4679-8dab-27505d87f5f2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:56.940326 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:56.941253 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:56.941566 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:56.941883 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] PATH_INFO: `/v3/projects/2793e3e3339d4334b05ffb0740597368/users/3f4a32199e4345acacf4231d6d026d55/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:56.960187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2793e3e3339d4334b05ffb0740597368, user_id=3f4a32199e4345acacf4231d6d026d55, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:56.963354 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:56.977238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1401/2792] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:01:56 2022] PUT /identity/v3/projects/2793e3e3339d4334b05ffb0740597368/users/3f4a32199e4345acacf4231d6d026d55/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:01:56.981453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:56.991417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f4981-2fd1-444d-98fd-6c7b045accb0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.037401 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:57.050271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcc2a5-d6d5-4fef-9dd5-86941790fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.051011 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcc2a5-d6d5-4fef-9dd5-86941790fd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.051297 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcc2a5-d6d5-4fef-9dd5-86941790fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.051574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcc2a5-d6d5-4fef-9dd5-86941790fd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.052294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcc2a5-d6d5-4fef-9dd5-86941790fd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.053932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcc2a5-d6d5-4fef-9dd5-86941790fd3d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.069514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1392/2793] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:57.074067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:57.085744 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d85f-2a58-45a1-8f34-4e6d51b17b42 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.090749 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.091627 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.091906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.092187 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] PATH_INFO: `/v3/projects/2793e3e3339d4334b05ffb0740597368/users/3f4a32199e4345acacf4231d6d026d55/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.094202 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2793e3e3339d4334b05ffb0740597368, user_id=3f4a32199e4345acacf4231d6d026d55, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.096983 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.107335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1402/2794] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:01:57 2022] PUT /identity/v3/projects/2793e3e3339d4334b05ffb0740597368/users/3f4a32199e4345acacf4231d6d026d55/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:57.118028 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.118326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.118612 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.147944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] MFA Rules not processed for user `3f4a32199e4345acacf4231d6d026d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:57.149400 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:57.227715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1393/2795] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:57.240361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:57.242076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:57.300682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89909bad-3229-4485-890a-d2248b02170c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.304569 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.305219 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.305506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.305790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.315361 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.316852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.326200 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1403/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:01:57.646054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:57.655181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151a780-e2c5-41ec-b33d-fae74f001e3d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.660458 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4a32199e4345acacf4231d6d026d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2793e3e3339d4334b05ffb0740597368', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.661116 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.661403 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.661689 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.662411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:01:57.662722 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.664342 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.676811 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1394/2797] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:01:57.680924 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:57.691080 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dce3e-bd11-496e-928e-5225d790557b service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.695920 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4a32199e4345acacf4231d6d026d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2793e3e3339d4334b05ffb0740597368', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.696634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.696930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.697239 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.697979 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.699996 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.715399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:57.735078 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1404/2798] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:01:57 2022] POST /identity/v3/projects => generated 352 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:57.740329 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:57.751060 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8904ac-c900-4ed1-b1d2-860adb608b63 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.809508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4a32199e4345acacf4231d6d026d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2793e3e3339d4334b05ffb0740597368', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.810075 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.810362 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.810646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.811378 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:01:57.811686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.813336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.826939 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1395/2799] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:01:57.863252 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:57.873644 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a2f57-9bc7-46e3-8814-1380e80f26e4 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:57.878697 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4a32199e4345acacf4231d6d026d55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2793e3e3339d4334b05ffb0740597368', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:57.879554 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:57.879839 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:57.880117 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] PATH_INFO: `/v3/projects/5cd4ae357b764f0a95710e3c4764904a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:57.881317 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5cd4ae357b764f0a95710e3c4764904a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:57.883206 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:57.931426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd4ae357b764f0a95710e3c4764904a', 'request_id': 'req-785d90d8-5dc5-44ba-810b-62a9bce569c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:57.939105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd4ae357b764f0a95710e3c4764904a', 'request_id': 'req-785d90d8-5dc5-44ba-810b-62a9bce569c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:57.947243 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd4ae357b764f0a95710e3c4764904a', 'request_id': 'req-785d90d8-5dc5-44ba-810b-62a9bce569c3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:57.958557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] The token cache is being invalidate because project 5cd4ae357b764f0a95710e3c4764904a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:57.958797 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:57.959437 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-785d90d8-5dc5-44ba-810b-62a9bce569c3 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:57.961198 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1405/2800] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:01:57 2022] DELETE /identity/v3/projects/5cd4ae357b764f0a95710e3c4764904a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:01:58.169188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:58.171020 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:58.231500 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:58.245686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e629b-9aa2-40c1-9010-882341ae4b07 tempest-VolumeQuotaClassesTest-639633200 tempest-VolumeQuotaClassesTest-639633200-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:58.304751 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:58.305631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:58.305914 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:58.306198 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] PATH_INFO: `/v3/users/fd025e99948246979c0046a74207ac7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:58.307394 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd025e99948246979c0046a74207ac7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:58.309372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:58.356271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd025e99948246979c0046a74207ac7d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.368139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd025e99948246979c0046a74207ac7d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.368935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd025e99948246979c0046a74207ac7d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.376452 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd025e99948246979c0046a74207ac7d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.377364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd025e99948246979c0046a74207ac7d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.389468 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1396/2801] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:01:58 2022] DELETE /identity/v3/users/fd025e99948246979c0046a74207ac7d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:58.394884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:58.395191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:58.395473 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:58.415778 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:58.417071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:58.488132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1406/2802] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:01:58.497861 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:58.499271 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:58.557454 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab04b-8e9d-4da7-baba-78d345ebf345 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:58.561146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:58.561793 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:58.562073 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:58.562349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:58.572091 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:58.573567 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:58.583464 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1397/2803] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:01:58.869231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:58.870767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:58.883536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c37e5-bde9-4844-8086-29a7d506eb7f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:58.888349 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:58.889213 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:58.889519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:58.889774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] PATH_INFO: `/v3/projects/8b5d5aeec557494bb95ac3e10171c1c9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:58.890973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5d5aeec557494bb95ac3e10171c1c9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:58.893678 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:58.942729 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5d5aeec557494bb95ac3e10171c1c9', 'request_id': 'req-8bb79828-473b-4879-ba81-1f3eb1f045ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.949352 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5d5aeec557494bb95ac3e10171c1c9', 'request_id': 'req-8bb79828-473b-4879-ba81-1f3eb1f045ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.955887 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5d5aeec557494bb95ac3e10171c1c9', 'request_id': 'req-8bb79828-473b-4879-ba81-1f3eb1f045ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.963789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:58.967953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] The token cache is being invalidate because project 8b5d5aeec557494bb95ac3e10171c1c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:58.968262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.969369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:58.971656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1407/2804] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:01:58 2022] DELETE /identity/v3/projects/8b5d5aeec557494bb95ac3e10171c1c9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:58.975702 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:58.985358 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a100e-d86c-455e-932c-d23858eff8f1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.004770 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:59.045212 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.046096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.046376 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.046684 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] PATH_INFO: `/v3/users/3f4a32199e4345acacf4231d6d026d55` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.047858 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4a32199e4345acacf4231d6d026d55)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.050437 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.095220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4a32199e4345acacf4231d6d026d55'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.106708 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4a32199e4345acacf4231d6d026d55'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.107367 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4a32199e4345acacf4231d6d026d55'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.115180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4a32199e4345acacf4231d6d026d55'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.115846 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1a2eb32-271a-4123-99b9-9dbcaeeb71c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4a32199e4345acacf4231d6d026d55'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.129256 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1398/2805] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:01:58 2022] DELETE /identity/v3/users/3f4a32199e4345acacf4231d6d026d55 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:59.407666 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.409821 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:59.423684 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79828-473b-4879-ba81-1f3eb1f045ce admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.487257 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.488031 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.488342 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.488634 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] PATH_INFO: `/v3/projects/2793e3e3339d4334b05ffb0740597368` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.489825 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2793e3e3339d4334b05ffb0740597368)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.491742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.537309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2793e3e3339d4334b05ffb0740597368', 'request_id': 'req-67fce356-66ce-43cb-9f31-69e4191a4201'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.545590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2793e3e3339d4334b05ffb0740597368', 'request_id': 'req-67fce356-66ce-43cb-9f31-69e4191a4201'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.553339 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2793e3e3339d4334b05ffb0740597368', 'request_id': 'req-67fce356-66ce-43cb-9f31-69e4191a4201'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.563645 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] The token cache is being invalidate because project 2793e3e3339d4334b05ffb0740597368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:01:59.563935 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.564852 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67fce356-66ce-43cb-9f31-69e4191a4201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:01:59.566425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1408/2806] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:01:59 2022] DELETE /identity/v3/projects/2793e3e3339d4334b05ffb0740597368 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:01:59.571262 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.571743 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.571843 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.590270 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:59.591501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:59.652404 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1399/2807] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:59.661241 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.661622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.661992 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.680032 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:01:59.681288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:01:59.689177 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1409/2808] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:01:59.693596 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.703384 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825d1e8-c090-46b3-832f-15258fbeb799 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.708831 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.709495 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.709810 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.710160 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.711087 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:01:59.711814 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.713935 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.726183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1400/2809] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:01:59.730728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.740407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5497a-3956-43fe-ac41-942ae1b9fa04 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.745085 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.745738 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.746027 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.746310 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.747030 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.749012 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.776436 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1410/2810] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:01:59 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:59.779263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.790836 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b80b81-bc2c-4b08-a45f-c2925cba6bd0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.821644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:01:59.855647 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.856326 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.856624 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.856899 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.857588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.859498 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.879733 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1401/2811] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:01:59 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:01:59.883927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.895091 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547e183-f12b-49d3-9dbd-dee8115c0dd8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.899770 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.900460 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.900746 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.901022 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.901748 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.903346 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.916482 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1411/2812] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:59.920021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.930677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ae6ce-7e13-48d6-8911-e46760f4c892 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.935264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993a479-c047-4168-a65f-290d8ebcd6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.935944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993a479-c047-4168-a65f-290d8ebcd6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.936207 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993a479-c047-4168-a65f-290d8ebcd6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.936516 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993a479-c047-4168-a65f-290d8ebcd6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.937233 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993a479-c047-4168-a65f-290d8ebcd6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.938821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993a479-c047-4168-a65f-290d8ebcd6bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:01:59.949417 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1402/2813] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:01:59.954467 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:01:59.965526 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d415a-6203-4a27-b737-6d8a49335621 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:01:59.970099 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:01:59.971813 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:01:59.972114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:01:59.972435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] PATH_INFO: `/v3/projects/da781dc4e7804591a640ab039b3039d5/users/1c1eac411db948688fbc0fb9eaec6b12/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:01:59.987147 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da781dc4e7804591a640ab039b3039d5, user_id=1c1eac411db948688fbc0fb9eaec6b12, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:01:59.989862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:00.002842 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1412/2814] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:01:59 2022] PUT /identity/v3/projects/da781dc4e7804591a640ab039b3039d5/users/1c1eac411db948688fbc0fb9eaec6b12/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:00.008042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12d460-d5cd-493c-8fc4-2b1c45a5afe3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:00.008364 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12d460-d5cd-493c-8fc4-2b1c45a5afe3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:00.008661 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12d460-d5cd-493c-8fc4-2b1c45a5afe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:00.027231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7e12d460-d5cd-493c-8fc4-2b1c45a5afe3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:00.028517 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e12d460-d5cd-493c-8fc4-2b1c45a5afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:00.087135 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1403/2815] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:00.096688 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:00.098392 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:00.131624 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:00.167585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caba37-d419-4a8a-843c-a688e4bd1a72 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:00.172059 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:00.172739 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:00.173023 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:00.173297 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:00.182548 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:00.183971 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:00.193714 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1413/2816] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:01.091312 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.091850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.091954 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.119541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] MFA Rules not processed for user `1c1eac411db948688fbc0fb9eaec6b12`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:01.120729 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:01.192870 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1404/2817] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:01.197338 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:01.207320 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397c739-e748-4c20-830d-e9e766190e52 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:01.212640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:01.213306 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.213710 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.214015 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.214727 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:01.216690 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:01.253754 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1414/2818] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:02:01 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:01.258515 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:01.269995 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56086f-ceae-4f9d-a582-bddf3fe0b1d7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:01.304508 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:01.328540 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:01.329219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.329506 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.329826 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.330524 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:01.332507 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:01.354999 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1405/2819] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:02:01 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:01.358492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:01.369786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee7714-2d45-42ed-883b-7243681269b7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:01.374393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:01.375057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.375344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.375621 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.376408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:01.378884 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:01.389424 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1415/2820] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:01.393559 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:01.403910 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99296-686d-45d9-a3ed-37483f32518f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:01.409987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:01.410884 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.411172 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.411456 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] PATH_INFO: `/v3/projects/23f7345c773a44eabd46b5c26fed8319/users/db8a35210dd14973b71426538bd4872e/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.427453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f7345c773a44eabd46b5c26fed8319, user_id=db8a35210dd14973b71426538bd4872e, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:01.430088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:01.443776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1406/2821] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:02:01 2022] PUT /identity/v3/projects/23f7345c773a44eabd46b5c26fed8319/users/db8a35210dd14973b71426538bd4872e/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:02:01.447965 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:01.458492 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4e10-61cc-4979-9545-2a8a1ef5a3ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:01.514599 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e900d35-8a09-4847-ae0e-fef1aafe5979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:01.515266 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900d35-8a09-4847-ae0e-fef1aafe5979 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.515552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900d35-8a09-4847-ae0e-fef1aafe5979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.515829 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900d35-8a09-4847-ae0e-fef1aafe5979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.516654 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e900d35-8a09-4847-ae0e-fef1aafe5979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:01.518335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e900d35-8a09-4847-ae0e-fef1aafe5979 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:01.528248 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1416/2822] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:01.532606 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:01.543582 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac507230-1a0e-463e-80f9-d9cd00ac005c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:01.548209 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:01.549844 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:01.550140 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:01.550420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] PATH_INFO: `/v3/projects/23f7345c773a44eabd46b5c26fed8319/users/db8a35210dd14973b71426538bd4872e/roles/2e85089f6c1a4bd1a54741d06e106e27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:01.552421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f7345c773a44eabd46b5c26fed8319, user_id=db8a35210dd14973b71426538bd4872e, role_id=2e85089f6c1a4bd1a54741d06e106e27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:01.555055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:01.565686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1407/2823] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:02:01 2022] PUT /identity/v3/projects/23f7345c773a44eabd46b5c26fed8319/users/db8a35210dd14973b71426538bd4872e/roles/2e85089f6c1a4bd1a54741d06e106e27 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:02.189277 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:02.190118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:02.190393 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:02.218343 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] MFA Rules not processed for user `db8a35210dd14973b71426538bd4872e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:02.219489 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:02.293161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1417/2824] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:02.305952 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:02.307683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:02.370370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7731be-2f41-4c50-88a1-2877e8c55bf1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:02.373997 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:02.374646 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:02.374934 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:02.375218 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:02.433375 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:02.434823 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:02.443821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1408/2825] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:02:03.789359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:03.790752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:03.803283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259307d-568a-4a4a-ada2-988922fab1d5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:03.806841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:03.807480 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:03.807765 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:03.808046 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:03.817906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:03.819347 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:03.828925 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1418/2826] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:12.439937 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:13.824659 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:16.950143 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:17.016397 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc09904-2551-4080-83e4-b22d64d2caa1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:17.021220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:17.022093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:17.022379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:17.022657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] PATH_INFO: `/v3/users/1c1eac411db948688fbc0fb9eaec6b12` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:17.023850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1eac411db948688fbc0fb9eaec6b12)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:17.025822 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:17.077765 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1eac411db948688fbc0fb9eaec6b12'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.089678 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1eac411db948688fbc0fb9eaec6b12'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.090354 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1eac411db948688fbc0fb9eaec6b12'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.097465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1eac411db948688fbc0fb9eaec6b12'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.098263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1eac411db948688fbc0fb9eaec6b12'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.110399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1409/2827] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:02:16 2022] DELETE /identity/v3/users/1c1eac411db948688fbc0fb9eaec6b12 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:17.289324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:17.302158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23826914-d5c3-4d19-89bb-028025f5ff3e service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:17.359950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:17.360861 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:17.361175 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:17.361461 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] PATH_INFO: `/v3/projects/da781dc4e7804591a640ab039b3039d5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:17.362676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da781dc4e7804591a640ab039b3039d5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:17.365388 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:17.417357 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da781dc4e7804591a640ab039b3039d5', 'request_id': 'req-9057cf00-53dd-42ff-9e10-be5488beafb9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.424711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da781dc4e7804591a640ab039b3039d5', 'request_id': 'req-9057cf00-53dd-42ff-9e10-be5488beafb9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.431631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da781dc4e7804591a640ab039b3039d5', 'request_id': 'req-9057cf00-53dd-42ff-9e10-be5488beafb9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.441711 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] The token cache is being invalidate because project da781dc4e7804591a640ab039b3039d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:02:17.442055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.442735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.444449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1419/2828] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:02:17 2022] DELETE /identity/v3/projects/da781dc4e7804591a640ab039b3039d5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:17.448440 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:17.449977 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:17.464249 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311581f-5d52-4c8b-8104-563ec8546fb3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:17.521560 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:17.522450 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:17.522738 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:17.523033 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] PATH_INFO: `/v3/users/db8a35210dd14973b71426538bd4872e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:17.524219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8a35210dd14973b71426538bd4872e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:17.526174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:17.547887 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:17.576144 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8a35210dd14973b71426538bd4872e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.588709 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8a35210dd14973b71426538bd4872e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.589386 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8a35210dd14973b71426538bd4872e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.596563 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8a35210dd14973b71426538bd4872e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.597251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-577f8049-f9e3-4961-9f73-a6fa9c8e0ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8a35210dd14973b71426538bd4872e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.609322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1410/2829] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:02:17 2022] DELETE /identity/v3/users/db8a35210dd14973b71426538bd4872e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:17.783673 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:17.785134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:17.797909 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf00-53dd-42ff-9e10-be5488beafb9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:17.860936 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:17.861801 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:17.862088 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:17.862369 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] PATH_INFO: `/v3/projects/23f7345c773a44eabd46b5c26fed8319` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:17.863552 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f7345c773a44eabd46b5c26fed8319)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:17.867401 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:17.906105 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:17.922921 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f7345c773a44eabd46b5c26fed8319', 'request_id': 'req-3a9050ec-4d14-4b17-96be-a929682709ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.931145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f7345c773a44eabd46b5c26fed8319', 'request_id': 'req-3a9050ec-4d14-4b17-96be-a929682709ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.937661 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f7345c773a44eabd46b5c26fed8319', 'request_id': 'req-3a9050ec-4d14-4b17-96be-a929682709ce'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.949114 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] The token cache is being invalidate because project 23f7345c773a44eabd46b5c26fed8319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:02:17.949324 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.949951 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3a9050ec-4d14-4b17-96be-a929682709ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:17.951506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1420/2830] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:02:17 2022] DELETE /identity/v3/projects/23f7345c773a44eabd46b5c26fed8319 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:17.956803 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:17.957731 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:17.958034 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:17.976671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:17.978448 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:18.040085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1411/2831] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:02:18.049131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.049413 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.049716 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.068592 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:18.069816 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:18.077911 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1421/2832] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:18.082117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.091882 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8a6da-b33b-41c8-8185-47b4e7686354 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.096388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.097040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.097322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.099601 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.100380 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:02:18.100701 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.102327 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.114442 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1412/2833] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:02:18.118895 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.128522 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b26e99-090b-457e-8032-45a1d05c6ac0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.133458 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.134110 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.134399 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.134680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.135371 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.137331 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.173328 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1422/2834] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:02:18 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:18.178164 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.188660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b876b8-b6f7-4c93-bd3f-c5390716f39c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.246005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.246289 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.246576 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.246857 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.247539 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.250111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.275313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1413/2835] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:02:18 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:18.279426 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.290425 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b26d8f-2d12-4a15-a4fa-cf62be8c8593 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.295145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.295810 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.296098 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.296419 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.297150 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.298760 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.309788 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1423/2836] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:18.313912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.323867 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33700006-15c7-4dd6-adcb-fcec2da0ec2f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.328447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195a38-d893-4c97-8a0a-e456568abbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.329102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195a38-d893-4c97-8a0a-e456568abbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.329388 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195a38-d893-4c97-8a0a-e456568abbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.329688 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195a38-d893-4c97-8a0a-e456568abbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.330421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195a38-d893-4c97-8a0a-e456568abbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.332021 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195a38-d893-4c97-8a0a-e456568abbeb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.343379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1414/2837] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:18.347481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.358055 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd005674-e93a-44e2-9050-9315de36d7dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.362735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.363620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.363989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.364276 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] PATH_INFO: `/v3/projects/c92a514c3c1b48c1824bb8c1038ecb2f/users/86c410e615624eeabb3c2bb0d90603d0/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.379478 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92a514c3c1b48c1824bb8c1038ecb2f, user_id=86c410e615624eeabb3c2bb0d90603d0, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.382299 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.395958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1424/2838] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:02:18 2022] PUT /identity/v3/projects/c92a514c3c1b48c1824bb8c1038ecb2f/users/86c410e615624eeabb3c2bb0d90603d0/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:18.406219 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.406509 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.406787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.414100 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:18.435332 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] MFA Rules not processed for user `86c410e615624eeabb3c2bb0d90603d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:18.436588 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:18.503594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1415/2839] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:18.514793 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:18.516602 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:18.597789 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4255-b88e-48c6-8768-966254cf1ec7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:18.602250 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985b3a-1423-4ef0-aaa3-75e9911e3dbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e8552f561ce4b4fa837f907a58816aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:18.602923 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985b3a-1423-4ef0-aaa3-75e9911e3dbb service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:18.603214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985b3a-1423-4ef0-aaa3-75e9911e3dbb service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:18.603499 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985b3a-1423-4ef0-aaa3-75e9911e3dbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:18.612095 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985b3a-1423-4ef0-aaa3-75e9911e3dbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:18.613581 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985b3a-1423-4ef0-aaa3-75e9911e3dbb service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:18.622521 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1425/2840] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:28.500416 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=30 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:28.618464 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=78,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:40.365925 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:40.432536 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565b5c4-3b12-4b81-84df-fbaae4b7a483 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:40.437040 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:40.437902 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:40.438188 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:40.438466 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] PATH_INFO: `/v3/users/86c410e615624eeabb3c2bb0d90603d0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:40.439671 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c410e615624eeabb3c2bb0d90603d0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:40.442259 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:40.491024 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c410e615624eeabb3c2bb0d90603d0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:40.503086 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c410e615624eeabb3c2bb0d90603d0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:40.503792 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c410e615624eeabb3c2bb0d90603d0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:40.511190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c410e615624eeabb3c2bb0d90603d0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:40.511943 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c410e615624eeabb3c2bb0d90603d0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:40.524554 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1416/2841] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:02:40 2022] DELETE /identity/v3/users/86c410e615624eeabb3c2bb0d90603d0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:40.529119 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78bd17-991b-48bf-a940-24698923412c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:40.529607 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78bd17-991b-48bf-a940-24698923412c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:40.529720 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78bd17-991b-48bf-a940-24698923412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:40.549662 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4b78bd17-991b-48bf-a940-24698923412c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:40.551448 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b78bd17-991b-48bf-a940-24698923412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:40.611927 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1426/2842] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:40.622788 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:40.624215 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:40.686493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dfe17-b9e5-491a-b8c1-6dad29885d05 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:40.690251 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287764fe-f457-4bba-b2cd-dbde1c52a13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:40.691174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287764fe-f457-4bba-b2cd-dbde1c52a13d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:40.691174 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287764fe-f457-4bba-b2cd-dbde1c52a13d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:40.691513 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287764fe-f457-4bba-b2cd-dbde1c52a13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:40.701985 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287764fe-f457-4bba-b2cd-dbde1c52a13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:40.703460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287764fe-f457-4bba-b2cd-dbde1c52a13d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:40.714294 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1417/2843] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:40.997782 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78bd17-991b-48bf-a940-24698923412c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:40.999616 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b78bd17-991b-48bf-a940-24698923412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:41.020173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78bd17-991b-48bf-a940-24698923412c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.027649 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.028653 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.029172 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.029497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] PATH_INFO: `/v3/projects/c92a514c3c1b48c1824bb8c1038ecb2f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.031041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92a514c3c1b48c1824bb8c1038ecb2f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.033435 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.082892 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92a514c3c1b48c1824bb8c1038ecb2f', 'request_id': 'req-897709b2-5665-4133-9ddf-5c630d2ed98b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:41.090899 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92a514c3c1b48c1824bb8c1038ecb2f', 'request_id': 'req-897709b2-5665-4133-9ddf-5c630d2ed98b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:41.097179 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92a514c3c1b48c1824bb8c1038ecb2f', 'request_id': 'req-897709b2-5665-4133-9ddf-5c630d2ed98b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:41.107930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] The token cache is being invalidate because project c92a514c3c1b48c1824bb8c1038ecb2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:02:41.108218 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:41.108922 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-897709b2-5665-4133-9ddf-5c630d2ed98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:41.110462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1427/2844] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:02:40 2022] DELETE /identity/v3/projects/c92a514c3c1b48c1824bb8c1038ecb2f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:41.116441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.116677 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.116960 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.135368 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:41.136602 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:41.153310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:41.201240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1418/2845] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:41.209583 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.209881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.210165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.229028 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:41.230251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:41.238230 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1428/2846] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:41.242703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.252682 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e5b0a-b5fe-4e6c-8c09-1a734442916d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.258395 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.259069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.259349 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.259631 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.260398 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:02:41.260715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.262334 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.277804 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1419/2847] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:02:41.282303 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.291779 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe4ea-dcf8-4138-b224-ea985f6194de None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.296582 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.297246 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.297534 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.297814 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.298509 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.300447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.327363 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1429/2848] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:02:41 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:41.331581 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.342574 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ddbae-d1c3-4c54-967b-e00e46c9ffb4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.402287 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.402956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.403148 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.403434 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.404142 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.406139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.428277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1420/2849] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:02:41 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:41.433838 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.445033 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d5d18-36c3-46ee-a5ff-8f765bac42d2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.450245 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.450943 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.451228 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.451506 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.452242 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.453886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.466084 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1430/2850] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:41.470465 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.480958 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84348123-8bab-476b-963e-0737e10c2320 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.485739 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bba2d-3795-4d74-958a-d4fe4aa63f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.486414 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bba2d-3795-4d74-958a-d4fe4aa63f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.486705 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bba2d-3795-4d74-958a-d4fe4aa63f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.486990 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bba2d-3795-4d74-958a-d4fe4aa63f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.487712 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bba2d-3795-4d74-958a-d4fe4aa63f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.489377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bba2d-3795-4d74-958a-d4fe4aa63f40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.499801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1421/2851] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:41.504606 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.514794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6ffc-ef07-40e1-b835-9fdf4f734128 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.519590 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.521222 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.521507 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.521785 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] PATH_INFO: `/v3/projects/58dcb5ee9e2645a4936625a256d6b162/users/77811f6bbe054fcc9ee94c8d0515e0ed/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.537309 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dcb5ee9e2645a4936625a256d6b162, user_id=77811f6bbe054fcc9ee94c8d0515e0ed, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.540017 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.553359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1431/2852] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:02:41 2022] PUT /identity/v3/projects/58dcb5ee9e2645a4936625a256d6b162/users/77811f6bbe054fcc9ee94c8d0515e0ed/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:41.563549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.563840 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.564121 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.594668 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] MFA Rules not processed for user `77811f6bbe054fcc9ee94c8d0515e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:41.595872 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:41.664054 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1422/2853] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:41.678134 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:41.679799 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:41.702958 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:41.742323 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8834a-c0ef-4ed7-9ca9-d27597bdf707 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:41.745997 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2732-e009-4ab4-af60-bfc6205a45c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:41.746646 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2732-e009-4ab4-af60-bfc6205a45c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:41.746931 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2732-e009-4ab4-af60-bfc6205a45c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:41.747207 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2732-e009-4ab4-af60-bfc6205a45c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:41.757070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f2732-e009-4ab4-af60-bfc6205a45c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:41.758573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f2732-e009-4ab4-af60-bfc6205a45c7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:41.768585 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1432/2854] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:02:50.185931 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:50.206304 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:50.253759 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a7100-9c0d-4a0b-952f-977800587bab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:50.258501 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:50.259408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:50.259697 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:50.259998 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] PATH_INFO: `/v3/users/77811f6bbe054fcc9ee94c8d0515e0ed` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:50.261295 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77811f6bbe054fcc9ee94c8d0515e0ed)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:50.263373 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:50.312897 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77811f6bbe054fcc9ee94c8d0515e0ed'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.325603 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77811f6bbe054fcc9ee94c8d0515e0ed'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.326290 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77811f6bbe054fcc9ee94c8d0515e0ed'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.334411 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77811f6bbe054fcc9ee94c8d0515e0ed'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.335045 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77811f6bbe054fcc9ee94c8d0515e0ed'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.347447 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1423/2855] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:02:50 2022] DELETE /identity/v3/users/77811f6bbe054fcc9ee94c8d0515e0ed => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:50.352158 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:50.352682 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:50.353034 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:50.371967 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:50.373671 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:50.435622 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1433/2856] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:50.446178 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:50.447570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:50.505093 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d6f38-b1dc-4ae7-8266-37d1b7e01da5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:50.509097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b27531-1cdb-4568-bdd8-6b48dc3b5618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:50.509630 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b27531-1cdb-4568-bdd8-6b48dc3b5618 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:50.509912 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b27531-1cdb-4568-bdd8-6b48dc3b5618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:50.510190 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b27531-1cdb-4568-bdd8-6b48dc3b5618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:50.520923 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b27531-1cdb-4568-bdd8-6b48dc3b5618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:50.522381 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b27531-1cdb-4568-bdd8-6b48dc3b5618 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:50.532199 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1424/2857] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:50.798260 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:50.800203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:50.813248 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b629-9908-4951-858d-df4447d9e50a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:50.818980 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:50.819855 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:50.820166 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:50.820488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] PATH_INFO: `/v3/projects/58dcb5ee9e2645a4936625a256d6b162` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:50.821881 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58dcb5ee9e2645a4936625a256d6b162)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:50.823912 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:50.871589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58dcb5ee9e2645a4936625a256d6b162', 'request_id': 'req-2ce63799-feec-4675-b90c-05eb14401681'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.879141 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58dcb5ee9e2645a4936625a256d6b162', 'request_id': 'req-2ce63799-feec-4675-b90c-05eb14401681'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.885629 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58dcb5ee9e2645a4936625a256d6b162', 'request_id': 'req-2ce63799-feec-4675-b90c-05eb14401681'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.897196 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] The token cache is being invalidate because project 58dcb5ee9e2645a4936625a256d6b162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:02:50.897481 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.898145 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2ce63799-feec-4675-b90c-05eb14401681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:50.899693 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1434/2858] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:02:50 2022] DELETE /identity/v3/projects/58dcb5ee9e2645a4936625a256d6b162 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:50.904944 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:50.905273 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:50.905561 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:50.927672 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:50.928929 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:50.966492 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:50.994068 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1425/2859] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:51.001938 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.002231 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.002553 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.022335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:51.023547 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:51.031447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1435/2860] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:51.035821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.045666 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92f1a0-4c1b-4f1b-95cf-36c065d1c63b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.050537 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.051484 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.051776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.052077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.052921 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:02:51.053280 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.054993 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.067703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1426/2861] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:02:51.071890 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.081370 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04020699-03be-4d51-ad9a-a89c2b3ee25c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.086901 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.087843 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.088131 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.088449 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.089157 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.091071 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.116680 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1436/2862] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:02:51 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:51.120719 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.131171 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80004741-d870-4710-8f78-533d6c927fa1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.189084 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.189920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.189920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.190161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.190852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.193505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.215220 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1427/2863] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:02:51 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:02:51.219574 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.230135 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034e6cf-a0f4-42ff-a881-c2c3668e8380 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.234790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.235459 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.235742 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.236021 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.236786 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.238731 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.249728 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1437/2864] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:51.254245 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.277331 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd900e07-dba7-495d-b060-b2906f3185dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.282104 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7f4d0-cd1c-4309-a0a2-909945eb3139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.282771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f4d0-cd1c-4309-a0a2-909945eb3139 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.283053 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f4d0-cd1c-4309-a0a2-909945eb3139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.283335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f4d0-cd1c-4309-a0a2-909945eb3139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.284055 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f4d0-cd1c-4309-a0a2-909945eb3139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.285768 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7f4d0-cd1c-4309-a0a2-909945eb3139 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.297455 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1428/2865] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:51.301874 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.312286 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1eb29-6cf7-4c2c-b2d3-236552e16274 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.318345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.319238 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.319527 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.319809 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] PATH_INFO: `/v3/projects/3d5fa4dfd8ea43e7b58290308bbfebea/users/0017800bbb9a450ab93c97c271fc8324/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.322335 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:51.338565 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5fa4dfd8ea43e7b58290308bbfebea, user_id=0017800bbb9a450ab93c97c271fc8324, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.341283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.354122 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1438/2866] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:02:51 2022] PUT /identity/v3/projects/3d5fa4dfd8ea43e7b58290308bbfebea/users/0017800bbb9a450ab93c97c271fc8324/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:51.416521 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.416808 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.417088 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.446314 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] MFA Rules not processed for user `0017800bbb9a450ab93c97c271fc8324`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:51.447577 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:51.510992 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1429/2867] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:02:51.522272 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:51.523998 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:51.581913 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b32c-8c98-4fae-9926-cf034a9a478f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:51.585871 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:51.586523 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:51.586811 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:51.587359 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:51.596156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:51.597665 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:51.606496 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1439/2868] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:53.671715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:53.673167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:53.714096 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:53.745422 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82168a30-cffa-46e4-b34b-be9aa1282511 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:53.750428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:53.751078 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:53.751372 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:53.751657 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:53.753460 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:53.765740 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:53.767336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6684a8-c70e-4d12-b00f-903dc4b8f85d service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:53.777546 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1430/2869] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:54.154140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:54.218431 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb494ef-394b-476d-800b-83fcc8417c95 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:54.223544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:54.224447 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:54.224735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:54.225010 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] PATH_INFO: `/v3/users/0017800bbb9a450ab93c97c271fc8324` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:54.226195 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0017800bbb9a450ab93c97c271fc8324)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:54.228113 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:54.274686 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0017800bbb9a450ab93c97c271fc8324'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.275330 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0017800bbb9a450ab93c97c271fc8324'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.282156 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0017800bbb9a450ab93c97c271fc8324'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.282780 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0017800bbb9a450ab93c97c271fc8324'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.291289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0017800bbb9a450ab93c97c271fc8324'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.309325 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1440/2870] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:02:54 2022] DELETE /identity/v3/users/0017800bbb9a450ab93c97c271fc8324 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:54.313812 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:54.314102 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:54.314385 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:54.332920 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:54.334132 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:54.395570 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1431/2871] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:54.405265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:54.406685 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:54.414351 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:54.469018 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a5e74-3a84-4420-a142-5cf866e5a6fe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:54.473234 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc907b-da64-44ba-aecb-743b47257f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:54.474118 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc907b-da64-44ba-aecb-743b47257f61 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:54.474407 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc907b-da64-44ba-aecb-743b47257f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:54.474689 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc907b-da64-44ba-aecb-743b47257f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:54.483732 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc907b-da64-44ba-aecb-743b47257f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:54.485212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc907b-da64-44ba-aecb-743b47257f61 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:54.494885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1441/2872] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:54.767545 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:54.769161 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:54.782519 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35c3fc-ebd5-4030-9455-657dc263a0c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:54.787829 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:54.788776 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:54.789063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:54.789340 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] PATH_INFO: `/v3/projects/3d5fa4dfd8ea43e7b58290308bbfebea` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:54.790703 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d5fa4dfd8ea43e7b58290308bbfebea)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:54.792717 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:54.844439 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d5fa4dfd8ea43e7b58290308bbfebea', 'request_id': 'req-47924e9b-e00c-449e-8ea7-548c67b5e26a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.850268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d5fa4dfd8ea43e7b58290308bbfebea', 'request_id': 'req-47924e9b-e00c-449e-8ea7-548c67b5e26a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.858197 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d5fa4dfd8ea43e7b58290308bbfebea', 'request_id': 'req-47924e9b-e00c-449e-8ea7-548c67b5e26a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.869763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] The token cache is being invalidate because project 3d5fa4dfd8ea43e7b58290308bbfebea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:02:54.870052 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.870690 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-47924e9b-e00c-449e-8ea7-548c67b5e26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:02:54.872226 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1432/2873] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:02:54 2022] DELETE /identity/v3/projects/3d5fa4dfd8ea43e7b58290308bbfebea => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:54.877161 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:54.877466 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:54.877737 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:54.896619 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:54.897824 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:54.960065 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1442/2874] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:54.968111 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:54.968462 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:54.968746 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:54.986710 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:54.987903 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:54.995636 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1433/2875] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:54.999758 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.010173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a343f60-31fb-4502-8d18-34f44cdfdc4a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.014628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.015289 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.015573 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.015853 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.016623 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:02:55.017049 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.018550 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.033203 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1443/2876] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:02:55.037761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.047030 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe32d55-443e-4aa3-98d7-7293d0c2b490 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.051795 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.052514 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.052797 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.053072 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.053769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.055667 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.085335 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1434/2877] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:02:55 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:55.090137 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.100259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa66d0-594b-450b-9dad-2a1937601f00 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.158263 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.158950 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.159214 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.159497 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.160194 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.162176 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.185993 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1444/2878] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:02:55 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:02:55.186979 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.197907 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3339a0-f6c6-47f1-abe6-b1ce435297e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.202595 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.203253 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.203541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.203890 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.204655 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.206264 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.217629 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1435/2879] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:55.221445 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.231510 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4a524-d600-417d-999a-067bc5707905 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.236192 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133710b-10de-4009-89f0-a327f4cfb254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.236885 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133710b-10de-4009-89f0-a327f4cfb254 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.237165 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133710b-10de-4009-89f0-a327f4cfb254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.237451 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133710b-10de-4009-89f0-a327f4cfb254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.238790 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133710b-10de-4009-89f0-a327f4cfb254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.240557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133710b-10de-4009-89f0-a327f4cfb254 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.250611 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1445/2880] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:02:55.255796 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.265820 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9100f-ee20-491c-b4bf-f9fd787392ef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.271082 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.271956 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.272240 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.272549 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] PATH_INFO: `/v3/projects/260fdce225db48d0b5a3205ecdce2f0c/users/22e89c62438b464fa49c08d606101ee9/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.275544 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:55.289146 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260fdce225db48d0b5a3205ecdce2f0c, user_id=22e89c62438b464fa49c08d606101ee9, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.291787 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.305693 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1436/2881] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:02:55 2022] PUT /identity/v3/projects/260fdce225db48d0b5a3205ecdce2f0c/users/22e89c62438b464fa49c08d606101ee9/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:02:55.310944 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e4937-20f4-4ddb-adef-d92f75ea9113 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.311233 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e4937-20f4-4ddb-adef-d92f75ea9113 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.311512 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e4937-20f4-4ddb-adef-d92f75ea9113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.330259 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4e5e4937-20f4-4ddb-adef-d92f75ea9113 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:55.331462 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4e5e4937-20f4-4ddb-adef-d92f75ea9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:55.340643 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:02:55.393267 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1446/2882] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:55.404621 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:55.407713 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:55.464723 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e799c98-8224-4e73-9fec-956fba196de5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:55.468474 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:55.469117 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:55.469399 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:55.469679 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:55.482726 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:55.484195 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:55.494159 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1437/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 14:02:56.150589 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf6ad1-1616-414e-8cf1-f3fc06e1be21 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:56.150886 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf6ad1-1616-414e-8cf1-f3fc06e1be21 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:56.151171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf6ad1-1616-414e-8cf1-f3fc06e1be21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:56.179376 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eebf6ad1-1616-414e-8cf1-f3fc06e1be21 None None] MFA Rules not processed for user `22e89c62438b464fa49c08d606101ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:56.180774 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eebf6ad1-1616-414e-8cf1-f3fc06e1be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:56.249800 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1447/2884] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:56.261681 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:56.263183 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:56.322575 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e0772-5835-4b07-9a12-68d0a359443e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:56.326428 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:56.327089 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:56.327379 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:56.327664 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:56.337687 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:56.339420 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:56.348407 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1438/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:57.724041 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:57.724656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:57.724656 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:57.742536 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] MFA Rules not processed for user `22e89c62438b464fa49c08d606101ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:02:57.743750 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:57.750631 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1448/2886] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:02:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:02:57.996745 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:57.998374 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:58.011371 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdc0f4-a8cb-4088-987e-349a01a5b83d service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:58.015231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:58.015930 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:58.016265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:58.016618 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:58.018445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:58.029832 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:58.031296 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:58.040322 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1439/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:02:58.355217 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:02:58.356769 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:58.439914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9ee88-19b9-4076-8e9c-10bdb2b60dcf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:02:58.445767 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:02:58.446415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:02:58.446696 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:02:58.446973 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:02:58.448804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:02:58.463591 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:02:58.465293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc0111-ce27-4d65-b864-449ee3935fcf service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:02:58.476456 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1449/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:02:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:03:07.178505 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:03:07.180042 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:07.265587 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577df71-8621-4e84-8bf4-9dddedfddf05 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:03:07.271235 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535f23db24704422a4128ff09f1cc284', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:03:07.271888 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:03:07.272177 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:03:07.272493 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:03:07.274336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:03:07.287181 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:03:07.288841 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:03:07.300854 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1440/2889] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:03:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:03:08.471275 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:03:17.295370 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:04:10.878070 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:04:10.878628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:04:10.878628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:04:10.896471 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] MFA Rules not processed for user `22e89c62438b464fa49c08d606101ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:04:10.898293 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:04:10.905262 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1450/2890] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:04:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:04:20.890999 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:13.458763 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:13.470231 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f0e1-a91b-4a05-ab4b-c22fb92f420a service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:13.477065 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:13.478208 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:13.478590 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:13.478801 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] PATH_INFO: `/v3/users/22e89c62438b464fa49c08d606101ee9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:13.480475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e89c62438b464fa49c08d606101ee9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:13.483022 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:13.531635 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e89c62438b464fa49c08d606101ee9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.543463 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e89c62438b464fa49c08d606101ee9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.544167 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e89c62438b464fa49c08d606101ee9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.551665 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e89c62438b464fa49c08d606101ee9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.552715 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c2c11d7b-41fe-4b9f-b92d-df3d4eb3c7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e89c62438b464fa49c08d606101ee9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.565932 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1441/2891] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:05:13 2022] DELETE /identity/v3/users/22e89c62438b464fa49c08d606101ee9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:13.737265 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:13.748905 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d18e58-d98c-47a8-b7a7-5c71b57162a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:13.807626 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:13.808518 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:13.808804 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:13.809079 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] PATH_INFO: `/v3/projects/260fdce225db48d0b5a3205ecdce2f0c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:13.810244 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] RBAC: Authorizing `identity:delete_project(project_id=260fdce225db48d0b5a3205ecdce2f0c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:13.812140 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:13.858906 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260fdce225db48d0b5a3205ecdce2f0c', 'request_id': 'req-44af0543-f1a5-4250-924c-abfcdf62572f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.866954 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260fdce225db48d0b5a3205ecdce2f0c', 'request_id': 'req-44af0543-f1a5-4250-924c-abfcdf62572f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.873251 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260fdce225db48d0b5a3205ecdce2f0c', 'request_id': 'req-44af0543-f1a5-4250-924c-abfcdf62572f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.884488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] The token cache is being invalidate because project 260fdce225db48d0b5a3205ecdce2f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:05:13.884982 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.885705 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44af0543-f1a5-4250-924c-abfcdf62572f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:13.887408 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1451/2892] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:05:13 2022] DELETE /identity/v3/projects/260fdce225db48d0b5a3205ecdce2f0c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:13.892686 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:13.892987 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:13.893277 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:13.912005 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:13.913265 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:13.974069 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1442/2893] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:13.981735 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:13.982025 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:13.982304 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.001345 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:14.002556 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:14.011102 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1452/2894] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 14:05:14.015306 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.025085 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ead019-89af-4ca3-82e7-a550f1edfa9e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.029613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.030268 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.030547 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.030821 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.031541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:05:14.031850 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.033475 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.045725 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1443/2895] 10.222.0.16 () {58 vars in 1152 bytes} [Thu May 26 14:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:05:14.050415 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.060640 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfe0a2-cd5c-488f-bbdd-c7518ef62330 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.065394 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.066057 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.066344 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.066630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.068076 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.070050 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.096232 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1453/2896] 10.222.0.16 () {60 vars in 1152 bytes} [Thu May 26 14:05:14 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:05:14.100600 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.110408 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456175-a938-4f96-8f94-a52d1c86d849 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.169377 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.170049 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.170336 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.170617 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.171311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.173274 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.180497 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:14.195737 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1444/2897] 10.222.0.16 () {60 vars in 1143 bytes} [Thu May 26 14:05:14 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:05:14.199953 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.210191 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fff7f-0d08-48e1-a602-eaa24643cd9a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.215013 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.215676 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.215961 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.216240 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.216986 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.218620 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.229712 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1454/2898] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:05:14.233660 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.244284 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abca42e-31c5-4c38-a483-03c328b25fc5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.248962 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6ff81-4bb6-438e-8b1c-85921f6580c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.249625 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6ff81-4bb6-438e-8b1c-85921f6580c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.249913 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6ff81-4bb6-438e-8b1c-85921f6580c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.250201 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6ff81-4bb6-438e-8b1c-85921f6580c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.250926 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6ff81-4bb6-438e-8b1c-85921f6580c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.252695 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6ff81-4bb6-438e-8b1c-85921f6580c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.263063 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1445/2899] 10.222.0.16 () {58 vars in 1121 bytes} [Thu May 26 14:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:05:14.267171 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.277468 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88811d68-67bc-4839-8ba1-dcb95edb1ad7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.282173 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.283058 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.283348 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.283628 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] PATH_INFO: `/v3/projects/615ac07094d843bd908799a4b5df6ec1/users/00e959015a944a86b57e4e1fd49b224e/roles/ee34d08904734afdb4939fd27d04b760` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.298862 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615ac07094d843bd908799a4b5df6ec1, user_id=00e959015a944a86b57e4e1fd49b224e, role_id=ee34d08904734afdb4939fd27d04b760)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.302197 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.315132 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1455/2900] 10.222.0.16 () {60 vars in 1482 bytes} [Thu May 26 14:05:14 2022] PUT /identity/v3/projects/615ac07094d843bd908799a4b5df6ec1/users/00e959015a944a86b57e4e1fd49b224e/roles/ee34d08904734afdb4939fd27d04b760 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:14.325180 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.325479 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.325769 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.353313 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] MFA Rules not processed for user `00e959015a944a86b57e4e1fd49b224e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:14.354613 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:14.425097 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1446/2901] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:05:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:14.436669 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:14.438428 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:14.470007 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:14.500283 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28a4d-addd-44cc-9ac5-cb73a7cfcc11 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:14.503841 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8214c82-6bdb-4348-a858-a8f320ca83d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '867e9b57fa7947e3afcbd974ed887969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:14.504515 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8214c82-6bdb-4348-a858-a8f320ca83d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:14.504794 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8214c82-6bdb-4348-a858-a8f320ca83d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:14.505067 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8214c82-6bdb-4348-a858-a8f320ca83d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:14.514859 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8214c82-6bdb-4348-a858-a8f320ca83d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:14.516519 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8214c82-6bdb-4348-a858-a8f320ca83d0 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:14.525930 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1456/2902] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:05:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:05:24.422179 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:24.521681 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=14 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:33.961750 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:34.025771 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d641ef-59b3-4f3e-9e6b-44f14d4e09b0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:34.030421 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:34.031311 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:34.031594 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:34.031876 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] PATH_INFO: `/v3/users/00e959015a944a86b57e4e1fd49b224e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:34.033139 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e959015a944a86b57e4e1fd49b224e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:34.035644 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:34.089975 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e959015a944a86b57e4e1fd49b224e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.105348 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e959015a944a86b57e4e1fd49b224e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.106131 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e959015a944a86b57e4e1fd49b224e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.114779 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e959015a944a86b57e4e1fd49b224e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.115441 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e959015a944a86b57e4e1fd49b224e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.129310 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1447/2903] 10.222.0.16 () {58 vars in 1223 bytes} [Thu May 26 14:05:33 2022] DELETE /identity/v3/users/00e959015a944a86b57e4e1fd49b224e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:34.133914 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:34.134479 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:34.134596 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:34.152630 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] MFA Rules not processed for user `8243f73b32484519bc2bed2bc9edc57e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:05:34.153863 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:34.216227 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1457/2904] 10.222.0.16 () {58 vars in 957 bytes} [Thu May 26 14:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 14:05:34.226734 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:34.228196 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:34.288569 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39a93-af17-463c-b47d-61884bd3c2a0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:34.292236 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f44d10be1654d17990966ae64d43ebd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:34.292924 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:34.293210 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:34.293488 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:34.303263 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:34.304761 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:34.314453 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1448/2905] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:05:34.598488 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:05:34.599989 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:05:34.614337 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdfe62-a64e-4d2b-b290-ebf141632da3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:05:34.619314 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8243f73b32484519bc2bed2bc9edc57e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56278f94460f422387fbeb96048d2270', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:05:34.620278 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:05:34.620752 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:05:34.621120 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] PATH_INFO: `/v3/projects/615ac07094d843bd908799a4b5df6ec1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:05:34.622544 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615ac07094d843bd908799a4b5df6ec1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:05:34.624557 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:05:34.675318 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615ac07094d843bd908799a4b5df6ec1', 'request_id': 'req-59115f5f-46d8-48cf-88c5-a7e04552a647'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.686148 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615ac07094d843bd908799a4b5df6ec1', 'request_id': 'req-59115f5f-46d8-48cf-88c5-a7e04552a647'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.694212 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615ac07094d843bd908799a4b5df6ec1', 'request_id': 'req-59115f5f-46d8-48cf-88c5-a7e04552a647'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.704288 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] The token cache is being invalidate because project 615ac07094d843bd908799a4b5df6ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 14:05:34.704609 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.705274 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59115f5f-46d8-48cf-88c5-a7e04552a647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 14:05:34.706873 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1458/2906] 10.222.0.16 () {58 vars in 1232 bytes} [Thu May 26 14:05:34 2022] DELETE /identity/v3/projects/615ac07094d843bd908799a4b5df6ec1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:05:44.310129 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:05:44.701139 dv-cinder-819668-4-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:07:58.693445 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:07:58.695077 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:58.835663 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7beb92-5027-45ec-9fdb-81c7ee09d59a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:07:58.894534 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f14da5405ccf4fdfaf91b1ec52ad3801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07dccb3049524e049519153329f36e8f', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:07:58.895184 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:07:58.895467 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:07:58.895747 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:07:58.897541 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:07:58.908504 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:07:58.910683 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fec022-ce9f-4afc-b540-54a3d5190760 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:07:58.921410 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1449/2907] 10.222.0.16 () {60 vars in 1365 bytes} [Thu May 26 14:07:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 14:08:08.916852 dv-cinder-819668-4-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}